From: Peter Szinek <peter@...>
Date: 2008-11-18T04:55:26+09:00
Subject: Re: How to extract links of a particular class type

--Apple-Mail-13--463368042
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

require 'rubygems'
require 'scrubyt'

google_data = Scrubyt::Extractor.define do
   fetch 'http://www.google.com/search?hl=en&q=gap+inc'

   link_title "//a[@class='l']", :write_text => true do
     link_url
   end
   next_page "Next", :limit => 3
end

output_file = open("google_results.txt", 'w') do |f|
   google_data.to_hash.each do |result|
     f.puts "#{result[:link_title]} - #{result[:link_url]}"
   end
end

produces:

Shop clothes for women, men, maternity, baby, and kids at gap.com ...  
- http://www.gap.com/
Gap Inc. - http://www.gapinc.com/
Gap Inc. - Careers - http://www.gapinc.com/public/Careers/careers.shtml
The Gap Inc. News - The New York Times - http://topics.nytimes.com/top/news/business/companies/gap_the_inc/index.html
Gap (clothing retailer) - Wikipedia, the free encyclopedia - http://en.wikipedia.org/wiki/Gap_(clothing)
GPS: Summary for GAP INC - Yahoo! Finance - http://finance.yahoo.com/q?s=gps
GPS - BloggingStocks - http://gps.bloggingstocks.com/
....
....



HTH,
Peter
___
http://www.rubyrailways.com
http://scrubyt.org
--Apple-Mail-13--463368042--