From: Alex Stahl <astahl@...5.com>
Date: 2010-12-03T05:04:38+09:00
Subject: Re: Screen scraping an aspx site with Mechanize

--=-P7RCYf6EUYTUmqyEP3B4
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Have you looked into nokogiri at all?  You can use mechanize for the
server interaction (GET, POST, etc), then parse the response
object's .body with nokogiri.

As long as you don't have to deliberately replicate a "click", this
would work fine (use Selenium if you actually need a  click event).
Otherwise, GETing and POSTing to the links produces the same results.
Make sense?

req = Mechanize.new
resp = req.get("/path/to/desired/url")
page = Nokogiri::HTML resp.body
link = page.xpath("//xpath/to/link")
resp = req.get(link)



________________________________________________________________________

Alex Stahl | Sr. Quality Engineer | hi5 Networks, Inc. | astahl@hi5.com
| m: 415.710.6961 

On Thu, 2010-12-02 at 11:19 -0600, Sofie Willander wrote:

> Hi,
> 
> I've been googleing for over a week now, but I can't find out how to
> screen scrape an aspx-site with Mechanize (and I must use mechanize).
> 
> The site (https://portal.if.se/kopforsakring/ProductList.aspx?linkID=4)
> contains a button with the text "Bil" that I want to click, and after
> that click the continue button (green one with the text "Forts辰tt").
> 
> I've tried using Firebug to find viewstate, but I don't know what to do
> with it once I've found it.
> 
> Am I one the right track?
> Can anybody help me?
> 

--=-P7RCYf6EUYTUmqyEP3B4--