From: Peter Szinek <peter@...>
Date: 2006-11-20T02:10:37+09:00
Subject: Re: Ruby screen scraping

OK,here is the full code:

require 'open-uri'
require 'htree'
require 'rexml/document'
require 'mysql'

url = "http://www.google.com/search?q=ruby"
results = []

open(url) {
 |page| page_content = page.read()
 doc = HTree(page_content).to_rexml
 doc.root.each_element('//a[@class="l"]') {
        |elem| results << elem.attribute('href').value }

 dbh = Mysql.real_connect("localhost", "peter", "****", "build")

 results.each do |result|
   dbh.query("INSERT INTO result VALUES ('#{result}')")
 end
}

HTH,

Peter
__
http://www.rubyrailways.com