From: Eric Christopherson <echristopherson@...>
Date: 2013-01-12T05:30:54+09:00
Subject: Re: Is there any fatser gem which can do the webpage scrapping fatsest way

On Fri, Jan 11, 2013 at 2:21 PM, Arup Rakshit <lists@ruby-forum.com> wrote:
> Hi,
>
> I have need to do the webpage automation,here also wanted to know how to
> do webpage scrapping.
>
> So any gem that can help me out in all context,

You have quite a few choices. It would narrow them down to know
whether the sites you want to automate and scrape use JavaScript for
much of their functionality.

If there is no JavaScript, or it's only used for a few cosmetic
touches, you can probably get by with Mechanize. Otherwise, there are
a lot of JavaScript-enabled ones. The one I see mentioned most often
in the Ruby context is Watir WebDriver. Others (off the top of my
head; I'm not sure whether they are Ruby or Ruby-compatible) include
Selenium WebDriver, PhantomJS, HTMLUnit, Akephalos... perhaps someone
could suggest more (or you can just google `ruby web scraping OR
automation OR testing`).

As for speed, I don't know how they compare. I'm pretty sure a
non-JavaScript solution like Mechanize would be the fastest; if you go
with a JavaScripty one, probably something headless (i.e. doesn't
require a running web browser) and based on V8 (the JS core in Chrome)
would be best. But again, maybe others could illuminate the
discussion.