From: gf <8si.greg@...>
Date: 2010-03-29T10:05:43+09:00
Subject: Re: Using Mechanize To Submit Forms

Mechanize can return a Nokogiri::HTML document when you do:

doc = page.parser

Using Nokogiri you can do something like:

doc.css('input[@class="negativeButton"]')

to retrieve an array of the input tags with a class of
"negativeButton". You can fine-tune the returned values as needed
using select() or reject() or by switching from .css() to .xpath().

You'll need to look at the contents of the chart() Javascript function
which is being called by the onclick() handler to figure out what the
add the parameter to, and that will give you the full URL to retrieve.

On Mar 28, 3:29 pm, Robert Molloy <r.mol...@gmail.com> wrote:
> Hi guys,
>
> I'm trying to use Mechanize with Ruby to scrape data from this site
> (http://www.tse.or.jp/tseHpFront/HPLCDS0101E.do). The first part of my
> task is trying to automate an initial search form, I can do this fine
> using the following code.
>
> require 'rubygems'
> require 'mechanize'
>
> agent = Mechanize.new
> page = agent.get('http://www.tse.or.jp/tseHpFront/HPLCDS0101E.do')
>
> search_form = page.form('HPLCDS0101Form')
> search_form.callJorEFlg = 1
> search_form['method'] = 'search'
> search_form.exeKind = 'HPLCDS0101E'
> search_form.searchListCount = 2500
> search_form.checkbox_with(:value => '001').check
> search_form.checkbox_with(:value => '002').check
> search_form.checkbox_with(:value => '004').check
>
> page = agent.submit(search_form)
> stockform = page.form('HPLCDS0301Form')
>
> stock = stockform.button_with(:value => 'Display of stock price')
>
> pp stock
>
> When I submit the form, the data I get returned contains over 2300 links
> of the type below. The onclick="chart('1378')" with the number is the
> stock ticker symbol that determines which stock is going to be opened
> when you click that particular link.
>
> <input type="button"  property="chart_button"  value="Display of stock
> price" onclick="chart('1378')" class="negativeButton" />
>
> Basically, I want to be able to click all the links of the above type,
> and save the returned data in a file.
>
> Can anyone help nudge me along here? Really appreciate it, thanks!
> --
> Posted viahttp://www.ruby-forum.com/.