From: Daniel Miessler <daniel@...>
Date: 2008-11-06T01:24:39+09:00
Subject: What's the Best Way to Mimic an HTTP Request?

I'm trying to write a tool that will take a domain as an argument and
make a request to http://onsamehost.com and then capture the list of
domains that share that same IP. I want to parse out those IPs and put
them into an array that I can print to a file later.

Here's the code I'm trying to use:

--
require 'net/http'
require 'uri'

PATH = '/query.jsp'
USERAGENT = 'Opera'
HOST = 'onsamehost.com'

@http = Net::HTTP.new(HOST, 80)

resp, data = @http.get2(PATH, {'User-Agent' => USERAGENT})

puts resp
puts data
--

The problem is that I keep getting a redirect
(#<Net::HTTPMovedPermanently:0xb7c35ffc>), which doesn't happen when I
make the request from a regular browser.

So I sniffed the regular request with wireshark, and a browser sends a
bunch of additional headers when it makes the request. Cookies,
referrer, etc.

Are any of these headers more necessary than others, and is there a
preferred way to send the headers using Ruby?

Thanks for any thoughts...
-- 
Posted via http://www.ruby-forum.com/.