From: Curt Hibbs <curt@...>
Date: 2005-08-06T01:24:55+09:00
Subject: Re: Ruby on CodeZoo

I'm forwarding this on behalf of Marc Hedlund from O'Reilly who was 
unable to post this himself:

You wrote:
 > It appears that they are spidering sites like RubyForge for info about
 > Ruby projects. Beyond that I don't know how they are operating.

We wrote a custom crawler that goes to various sites (Sourceforge,
Freshmeat, CodeHaus, Parnassus, CheeseShop, RubyForge, etc.) and
collects as much information as we can about projects for developers in
our target language.  We then manually review each project.  If the
project is a component (code for other code to call) or a development
utility (IDE, etc.), if it has *something* on a user-facing home page
and seems at least plausibly documented, if the releases are not all
clustered in a 1-2 month period, but have come out over time, and if it
seems to serve a real and worthwhile purpose, then we include it on
CodeZoo.  If one or more of those conditions are not met, we either
keep checking back to see progress on the project, or reject it if it
seem wrong for our site.

 > I did a quick test by downloading FreeRIDE from their site, and it
 > seems
 > like they are serving it themselves (I couldn't detect any forward of
 > the request to RubyForge).

Correct -- we serve all downloads from our site directly.  Mainly, this
is a way to get around the massive package list and mirror selection
pain at Sourceforge.  We may at some point require people to be logged
in in order to download, but our purpose is to provide one-click
download, and very clear identification of the "main" and current
package file, for every project.

 > This is a very good thing because we needed a place where we could rate
 > and comment on Ruby packages.

Great, I'm glad to hear you think so.  If you have any suggestions for
how the site could work better, or how it could better integrate with
RubyGems, I'd be very interested to hear them.

 > Does anyone know they mechanics behind CodeZoo -- how is it acquiring
 > its information and keeping it up to date?

If I've left any questions unanswered, please let me know.

Thanks again,
Marc Hedlund
<marc@oreilly.com>