From: brabuhr@...
Date: 2008-12-21T09:19:03+09:00
Subject: Re: Anyone scraping dynamic AJAX sites?

On Sat, Dec 20, 2008 at 7:27 AM, Will Simpson <will1@wjrs.co.uk> wrote:
>>>
>>> [1]: Preferably a virtal one on a console-only machine.
>>> [2]: Sadly, afaik, firefox has no hidden-mode.


I've never used it, but Celerity appears to have Javascript support:

http://celerity.rubyforge.org/

> You could try using a virtual frame buffer if you are using Linux or
> similar.
>
> Xfvb :99 -ac &
> export DISPLAY=:99

Or, start a vncserver with xstartup set to launch the scraper script.