From: "Ball, Donald A Jr (Library)" <donald.ball@...>
Date: 2007-06-13T04:03:54+09:00
Subject: Re: Stuck in a Redirect Loop While Crawling

> Here is what FireFox is sending:
> 
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:
> 1.8.1.4) Gecko/20070515 Firefox/2.0.0.4
> Keep-Alive: 300
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Accept-Language: en-us,en;q=0.5
> Cookie: sessionid=6d7dd6277ec64983bf642760d7d77d6a
> Connection: keep-alive
> Accept: text/xml,application/xml,application/xhtml+xml,text/
> html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> Host: <hostname here>

Presumably this is from LiveHTTPHeaders? I note that the Referer header
is not included herein, but Firefox does send those data by default.
Perhaps that's the substantive difference between the Firefox request
and the Net::HTTP request? Just a thought.

- donald