From: Ammar Ali <ammarabuali@...>
Date: 2010-11-23T22:02:14+09:00
Subject: Re: nokogiri ip ban?

--0016e6d625f88d5c070495b7f2cc
Content-Type: text/plain; charset=UTF-8

On Tue, Nov 23, 2010 at 2:46 PM, Luis G. <l17339@gmail.com> wrote:

> Anyway, now I'm using an empty user agent, but I tried to define a user
> agent but the result was the same. I tried something like:
>
> Nokogiri::HTML(open(url, "User-Agent" => "Ruby/#{RUBY_VERSION}"))
>
> I also tried the user agents we can use in Mechanize ('Linux Mozilla',
> for example) but nothing worked.



It was just a guess. But, as you mentioned, your IP is being blocked, so
it's too late to change agents now. You may have been blocked for any reason
really, frequency of requests, user-agent, or something else entirely.
Usually such blocks are temporary (it could be a dynamic IP) so you could
try again later. But who knows how long it will take, or if you will be
blocked again.

Andrea's suggestion is probably your best bet, contact the owners of the
site and request access. You might find out why you got blocked and avoid it
in the future.

Regards,
Ammar

--0016e6d625f88d5c070495b7f2cc--