From: Ryan Mckenzie <ryan@...>
Date: 2010-09-04T23:24:59+09:00
Subject: Using Nokogiri to scrape multiple websites

Hi everyone,

I'm looking to use Nokogiri to scrape about 10 websites for their anchor
texts and output them on screen. What would be the best way to achieve
this?

I have tried doing something like this without much luck...

def index
  sites = Array.new("site1.com","site2.com","site3.com")
  sites.each do |site|
    @textlinks << scrape(site)
  end
end

def scrape(website)
  require 'open-uri'
  require 'nokogiri'

    doc = Nokogiri::HTML(open(website))
  return doc.xpath('//a')
  end

Thanks
-- 
Posted via http://www.ruby-forum.com/.