From: Andy Pipes <mypipeline@...>
Date: 2008-10-07T06:02:15+09:00
Subject: Mechanize select list help...?

Hi.

I'm using the excellent WWW::Mechanize to screen scrape a site for UK
frost dates (don't ask ;)

there's a lot of issues with the HTML not being grand, so I thought
that's where I am going wrong in my code, but I'd be really grateful if
somebody could give me a steer on this as I've been trying for hours,
and the documentation only gets me half-way :)

Here's the code. All I want to do is select each of the 100 or so towns
in the select list, follow the link via the submit button and scrape the
first and last frost dates from the resulting page.

Here's the code:

require 'rubygems'
require 'mechanize'

agent = WWW::Mechanize.new
page = agent.get('http://www.gardenaction.co.uk/main/weather1.asp')


town_results = page.form_with(:action => 'create_cookie.asp') do |e|
  e.fields.name('Town').options.each do |s|
    s.select
  end
end.submit

p town_results.search("/<p align=\"left\">HOME TOWN:(.*)<Form
Method=Post Action=\"create_cookie.asp\">/")

I think I'm actually getting as a result the page itself back not the
results page (which should be
http://www.gardenaction.co.uk/main/weather1-results.asp)

Can anyone give me some advice here? It should be obvious I'm new to
Ruby and OO so am fully expecting to have gone wrong here with instance
variables or the like :)

thanks in advance.

andy
-- 
Posted via http://www.ruby-forum.com/.