From: "David G. Andersen" <dga@...>
Date: 2004-10-27T10:26:25+09:00
Subject: Re: Server Programming

On Tue, Oct 26, 2004 at 11:04:07AM +0900, Andy Stone scribed:
> Would you say it's slower in processing text files?  A single file can
> run a few megabytes. That's where I was hoping to gain some speed from
> Ruby.  I was thinking that Ruby would be a little slower in other
> aspects, but thought that I would make up considerable ground with the
> parsing process.

Another view on this:

I just rewrote a lot of data processing scripts in Ruby.  The
prior version was a mix of perl with a few speed-critical
functions implemented as standalone C programs.  I _do_
care about speed with these -- a full analysis run on the
data we've got right now takes about a day and a half -- but
I also cared a lot about getting the programs done in time
for a conference deadline.  The data is about 32 gigs gzipped.

The new version is about the same speed as the prior version, or
just a bit slower, because it's done right, and it was easier to
do right -- but doing line processing with Ruby 1.8 (which doesn't
have the new oniguruma regexp library) is noticably slower than
doing the same thing in perl if you're doing a lot of

  if (l =~ /pattern/) ...

Doing it right resulted in the memory footprint dropping from
about 500MB to 7MB, which was pleasant, and avoiding some
stupidity in the perl code (which was pretty entrenched)
sped things up enough to compensate for the slowdowns. 
I'm _very_ glad I took the time to rewrite it.  I had to
do a few workarounds to avoid particular slowdowns in 
Ruby (see my earlier thread about zlib slowness, in particular),
but it wasn't onerous.

But if you ignore programmer effort and the possibility
of better program structure, comparing the raw speed of
a JITted Java to Ruby is probably going to come down on the
side of Java.  Of course, if you can easily implement your
"inner loop" functionality in C, you could always quickly
swig it and 'require' it from within ruby. :)

Interested to hear what you think after the project.

  -Dave

-- 
work: dga@lcs.mit.edu                          me:  dga@pobox.com
      MIT Laboratory for Computer Science           http://www.angio.net/