From: Michel Martens <blaumag@...>
Date: 2005-09-12T23:49:05+09:00
Subject: Re: website screen scraping with Mechanize or Rubyful Soup

Hi

On 9/12/05, Dan Kohn <dan@dankohn.com> wrote:
> I'm trying to get some website screen scraping working, but I'm
> suffering from a lack of examples and documentation for either
> WWW::Mechanize or Rubyful Soup.
> 
> With WWW::Mechanize, the only example I found was
> http://www.zenspider.com/pipermail/ruby/2005-July/002068.html.  I tried
> to simplify this to the script below, but it just prints out "My wife
> is ".
> 
> Rubyful Soup <http://www.crummy.com/software/RubyfulSoup/> also seems
> like a great library, but there doesn't seem to be a single example
> (only Python ones
> <http://www.crummy.com/software/BeautifulSoup/examples.html>).
> 
> #!/usr/bin/env ruby
> 
> require 'mechanize'
> 
> agent = WWW::Mechanize.new
> agent.user_agent_alias = 'Windows IE 6'
> 
> # get first page
> page = agent.get('http://www.dankohn.com/')
> md = page.body.match /My wife, (\w+\s\w+)<\/a>/m
> 
> printf "My wife is ", md

print "My wife is ", md[1]



Michel.