From: Nathan Taylor-Hoover Date: 2007-06-14T23:21:04+09:00 Subject: Re: mechanize newbie ------=_Part_41413_4564997.1181830865221 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Colin This line here form.vb_login_md5password = "password" Shows the form password value as being an md5password. Are you creating an md5 hash out of your password before supplying it to mechanize? If not then I would assume that is the problem. On 6/13/07, jfry wrote: > > Hi Colin, I can't tell you how to do it in mechanize, but I can say > that what you are trying to do is super easy in Watir: > http://openqa.org/watir > > Watir (Web Application Testing In Ruby) is primarily used for driving > browser-based test automation, but it has a wonderful API that makes > what you describe very easy. Originally the only choice of browser to > drive was IE, but now the FireWatir and SafariWatir projects are > getting strong as well. > > Best of luck, whatever solution you go with, > Jeff > > On Jun 13, 2:53 pm, "Colin Summers" wrote: > > Okay, Ruby in general newbie, but I did the whole shovell project for > > RoR, so I felt I was getting somewhere... > > > > I am fooling around trying to make a spider (scraper?) to pull content > > off the Forum I read all the time so that I can read it offline. > > > > It seemed like mechanize is exactly what I want. But I try this: > > > > require 'rubygems'; require 'mechanize' > > > > agent = WWW::Mechanize.new > > page = agent.get('http://dapo.org/forums/index.php') > > > > pp page > > > > puts "\n\n trying to login... \n\n" > > > > # Fill out the login form > > form = page.forms.first > > form.vb_login_username = "username" > > form.vb_login_md5password = "password" > > form.do ="login" > > form.s = "" > > > > page = agent.submit(form) > > > > pp page > > > > # pull down a thread > > page = agent.get('http://dapo.org/forums/archive/index.php?t-2293.html') > > > > pp page > > > > And it doesn't login (blank page for that last get). Clues? > > > > Thanks, > > --Colin > > > > ------=_Part_41413_4564997.1181830865221--