From: gf <8si.greg@...> Date: 2008-11-30T10:02:33+09:00 Subject: Re: Anyone scraping dynamic AJAX sites? The problem is you need a DOM-aware Javascript interpreter in your code to execute the Javascript, manipulate the DOM in the HTML, and then allow you to extract the data you need. There are projects like Rhino, which is a Javascript engine you can embed in other apps, but you still won't have the DOM of the page nor will you be able to manipulate it then extract the values, at least as far as I understand. You could use something like Ruby driving some sort of WebKit interface on Mac OS or Linux, but I have no idea where to start. That, to me, seems like the best answer. Maybe even a Ruby-based Cocoa app would be the trick. On Nov 29, 5:25 pm, Becca Girl wrote: > Hello. > > Is there anyone who has successfully found a way to scrape a dynamically > generated AJAX web site?  If I view the source, it gives me the > variables.  If I use Firebug to view the DOM, it gives me the actual > values.  Any ideas? > > Thanks. > -- > Posted viahttp://www.ruby-forum.com/.