From: Luis Parravicini <lparravi@...>
Date: 2009-03-27T01:05:46+09:00
Subject: Re: 400 "Bad Request"

Hi

On Thu, Mar 26, 2009 at 12:21 PM, Arun Kumar
<arunkumar@innovaturelabs.com> wrote:
> Hi,
> I'm developing a program to fetch the html contents of a site using
> 'net/http'. Everyting works fine except for http://www.youtube.com. When
> i pass that url an error like this is found
>
> /usr/lib/ruby/1.8/net/http.rb:2097:in `error!': 400 "Bad Request"
> (Net::HTTPServerException)
>
> I think it is because i'm not using any user agent. Can any body please
> tell me any auggestion. I'll be really greatful. This is my code
> snippet.


You can try getting the page with Mechanize [1].

irb -rubygems -rmechanize
irb(main):001:0> agent=WWW::Mechanize.new
irb(main):002:0> agent.get('http://www.youtube.com').code
=> "200"

Anyway, it seems it's as you said, without a User-Agent, youtube returns a 400:
irb(main):003:0> agent.user_agent=nil
=> nil
irb(main):004:0> agent.get('http://www.youtube.com').code
WWW::Mechanize::ResponseCodeError: 400 => Net::HTTPBadRequest
	from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.2/lib/www/mechanize.rb:229:in
`get'
	from (irb):4
irb(main):005:0>


[1] http://mechanize.rubyforge.org/

-- 
Luis Parravicini
http://ktulu.com.ar/blog/