From: Chuck Dawit <chuckdawit@...>
Date: 2007-09-21T05:26:08+09:00
Subject: Re: scraping web pages for cisco products

unknown wrote:
> On 9/19/07, Chuck Dawit <chuckdawit@gmail.com> wrote:
>> One idea I was given was to
>> split the pages into ones with forms and those without forms. Those
>> without forms probably wont have anything for sale so I can eliminate
>> those. But then I really don't know how to handle after that.
> 
> Here's a naive implementation of binning by forms:
> 

> 
>   page = agent.get "http://#{site}"
>   forms = page.forms
>   search_forms = forms.select{|f|
>     (f.name   and f.name.match        /search/i) or
>     (f.action and f.action.to_s.match /search/i)
>   }
> 
>   if search_forms.size > 0
>     bin1 << site
>   elsif forms.size > 0
>     bin2 << site
>   else
>     bin3 << site
>   end
> end
> 

I'm checking the size of the form like in the code above but when it
gets to the 13th url to check the script just exits. Does anyone know
why? How can I run a check on this?
-- 
Posted via http://www.ruby-forum.com/.