From: Peter Szinek <peter@...>
Date: 2008-11-30T22:28:52+09:00
Subject: Re: Anyone scraping dynamic AJAX sites?

--Apple-Mail-1-636747827
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

scRUBYt! - http://scrubyt.org

e.g. scraping your linkedin contacts:

require 'rubygems'
require 'scrubyt'


property_data = Scrubyt::Extractor.define :agent => :firefox do

   fetch          'https://www.linkedin.com/secure/login'
   fill_textfield 'session_key', '****'
   fill_textfield 'session_password', '****'
   submit

   click_link_and_wait 'Connections', 5

   vcard "//li[@class='vcard']" do
     first_name  "//span[@class='given-name']"
     second_name "//span[@class='family-name']"
     email       "//a[@class='email']"
   end

end

puts property_data.to_xml

Cheers,
Peter
___
http://www.rubyrailways.com
http://scrubyt.org


On 2008.11.30., at 1:25, Becca Girl wrote:

> Hello.
>
> Is there anyone who has successfully found a way to scrape a  
> dynamically
> generated AJAX web site?  If I view the source, it gives me the
> variables.  If I use Firebug to view the DOM, it gives me the actual
> values.  Any ideas?
>
> Thanks.
> -- 
> Posted via http://www.ruby-forum.com/.
>


--Apple-Mail-1-636747827--