From: sto.mar@...
Date: 2012-10-02T22:55:04+09:00
Subject: Re: Ruby Mechanize and do-if loops

Am 02.10.2012 15:19, schrieb Sybren Kooistra:
> Hi all,
>
> This is probably pretty basic stuff, but I can't seem to manage to get
> my head 'round it:
>
> I'm going through a number of webpages using mechanize. It is a
> database, with many pages and 10 links on each page. I save the first 10
> links into a txt file, continue to the next page, and save those 10.
> Etcetera.
>
> However, how do I determine it to to automatically stop when there is no
> 'next page' (link_with(:text => 'Next >') ?
>
> Code:
> require 'nokogiri'
> require 'open-uri'
> require 'mechanize'
> agent = Mechanize.new
> agent.get("http://www.miljoenhuizen.nl/houses/nl/1000/0/99999999/-/pg1?searchtype=archive")
>
> # get all links on the page and save to txt file
> pagina = agent.page
> links = pagina.search("//td[@class = 'upper']/a")
> open('pc1000.txt', 'a') { |f| f.puts links }
>
> #open next page and save to textfile
> agent.page.link_with(:text => 'Next >').click
> open('pc1000.txt', 'a') { |f| f.puts links }
>
> #etcetera, but how do make it loop and end untill there is no longer a
> 'Next >' link? to click ???!
>

link_with(:text => 'some text') returns nil when there is no such link.
You could use that in a loop:

loop do
   # do what you want to do with each page

   next_link = agent.page.link_with(:text => 'Next >')
   break  unless next_link
   next_link.click
end


-- 
<https://github.com/stomar/>