From: Dan Kohn <dan@...>
Date: 2005-09-12T15:56:32+09:00
Subject: website screen scraping with Mechanize or Rubyful Soup

I'm trying to get some website screen scraping working, but I'm
suffering from a lack of examples and documentation for either
WWW::Mechanize or Rubyful Soup.

With WWW::Mechanize, the only example I found was
http://www.zenspider.com/pipermail/ruby/2005-July/002068.html.  I tried
to simplify this to the script below, but it just prints out "My wife
is ".

Rubyful Soup <http://www.crummy.com/software/RubyfulSoup/> also seems
like a great library, but there doesn't seem to be a single example
(only Python ones
<http://www.crummy.com/software/BeautifulSoup/examples.html>).

#!/usr/bin/env ruby

require 'mechanize'

agent = WWW::Mechanize.new
agent.user_agent_alias = 'Windows IE 6'

# get first page
page = agent.get('http://www.dankohn.com/')
md = page.body.match /My wife, (\w+\s\w+)<\/a>/m

printf "My wife is ", md




Thanks in advance for any help you can offer.