From: Gene Tani <gene.tani@...>
Date: 2005-07-01T15:25:42+09:00
Subject: Re: spidering a website to build a sitemap

Try it: you'll get Errno::ECONNREFUSED (Net::HTTP) or it will time out
(open-uri) on a lot of large commercial websites, like ruby-lang.org.

So either I have to rewrite headers to emulate, say, Mozilla browser,
or throttle down number of GETs it's firing out, so as not to offend
the websites' firewalls.  Not clear from stdlib doc link in Marcus'
post.