From: Justin Collins <collinsj@...>
Date: 2006-02-01T04:45:24+09:00
Subject: Re: Looking for web crawler written in Ruby

Kev wrote:
> Hi, I am looking for web crawler(spider) written in Ruby. I googled but
> couldn't find much information. Can anyone give me some links?
>
> Thanks in advance.
>
> Kevin
>   

If you are _really_ desperate, the first Ruby program I ever wrote was 
kind of a web crawler: 
http://students.seattleu.edu/collinsj/programs_netcrawler.html

But it's really terrible. It isn't hard to write your own, if you need 
to, using Net::HTTP 
(http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/index.html) and URI 
(http://ruby-doc.org/stdlib/libdoc/uri/rdoc/index.html).

-Justin