From: "jfister@..." <jfister@...>
Date: 2005-11-02T01:07:06+09:00
Subject: Re: webcrawler/robot

There's one in the Rails Analyzer project that crawls a single domain:

RDoc:
http://rails-analyzer.rubyforge.org/hacks/

Article:
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/166fa4076defb055