From: Colin Summers <bladenut@...>
Date: 2007-06-14T06:53:37+09:00
Subject: mechanize newbie

Okay, Ruby in general newbie, but I did the whole shovell project for
RoR, so I felt I was getting somewhere...

I am fooling around trying to make a spider (scraper?) to pull content
off the Forum I read all the time so that I can read it offline.

It seemed like mechanize is exactly what I want. But I try this:

require 'rubygems'; require 'mechanize'

agent = WWW::Mechanize.new
page = agent.get('http://dapo.org/forums/index.php')

pp page

puts "\n\n trying to login... \n\n"

# Fill out the login form
form  = page.forms.first
form.vb_login_username = "username"
form.vb_login_md5password = "password"
form.do ="login"
form.s = ""

page  = agent.submit(form)

pp page

# pull down a thread
page = agent.get('http://dapo.org/forums/archive/index.php?t-2293.html')

pp page

And it doesn't login (blank page for that last get). Clues?

Thanks,
--Colin