From: Vikhyat Korrapati <vikhyatk@...>
Date: 2009-04-30T17:14:33+09:00
Subject: Re: Emulating a web browser

On Thu, 2009-04-30 at 16:54 +0900, Adam Bender wrote:
> I am looking for a library to help me emulate a web browser, at least at the
> network level.  By this I mean I would like to run a program that, from the
> point of view of a web server, behaves just like, say, Firefox, but I don't
> care about actually displaying text or images or anything like that.  What I
> would like it to do is speak HTTP, store and send cookies, automatically
> fetch embedded content like images and style sheets, and so forth.  I
> thought Mechanize was what I wanted, but it doesn't fetch embedded content.
> It doesn't even recognize it.  I could perhaps tell Nokogiri to find all the
> images and have Mechanize fetch them, but I've never used Nokogiri before, I
> don't know an exhaustive list of types of embedded content Firefox loads
> automatically (images, JavaScript, Flash, anything else?), and it seems like
> getting Mechanize to emulate FF's HTTP request for these objects is
> difficult.
> 
> Are there libraries that are meant for this type of interaction with
> websites?  Perhaps I'm better off abandoning Ruby and making a Firefox
> extension.

I'm not sure what you want to do, but have you looked at Watir?
http://wtr.rubyforge.org/

> Thanks,
> 
> Adam
-- 
Vikhyat Korrapati
http://aetus.net/