From: Douglas Baskins <dougbaskins@...>
Date: 2002-08-28T05:39:47+09:00
Subject: Re: ANN: RJudy-0.1 - Judy Arrays for Ruby

> [Joe]
> Thanks Lyle!
> 
> It works great. I find that insertion time is faster using judy,
> retrieval time is about the same. I also find that memory usage is
> lower with judy. Here is reading 1,000,000 unique words (12MB of
> data):
> 
> JudySL takes 23MB
> JudyHash takes 26MB
> RubyHash takes 40MB
> 
> (I subtracted the baseline process size of 54MB that the words array
> and ruby itself used).
> 
> Another datapoint:
> Time to insert 1000000 words into a JudySL array: 6907858.0 usec.
> 
> Doing the same with a straight C program took 3 seconds.  I think the
> extra 3.9 seconds is spent in rubyland doing this:
> 
> words.each do |word|
>     array[word] = word
> end
> 
> All in all, this is *super-cool*!

My thanks too, Lyle,

Joe:

I would be interested in knowing a little more about your data, such
as average string length.  I am presently looking into tuning JudySL
for shorter length string  (~1..15 chars).  I am trying to get hold
of some 'typical' data to do comparisons.  BTW, try long strings such
as a sentence and, of course, run yourself out of memory.  This is 
where JudySL does well compared with every thing else.

I am also looking to see why so many people are having trouble porting
JudySL to other machines.  The basic problem is the need for the
define -DJU_LINUX in the compile line (even after all the stuff 
under it is removed from JudySL.c) because it is used in JudyPrivate.h 
to specify that it is a little "Endian" (Intel/Dec) machine.
Big Endian machines to not need the byte swapping code/instruction.

Please, no flames, I know its a bad way to program.  We had a 2 week
window to get Judy into Open Source, or maybe loose the opportunity
altogether.  So it went out with just the 6 platforms and a hacked up 
./configure script.  I/we will fix Judy to be as platform independent
as possible soon.  Then I/we can get it on all platforms with a little
twik hear and there.

BTW, there is nothing inherent in Judy that is specific to anything
until you do the final hardware/compiler tunes -- such as some 
have instruction support for some of the algorithms used in Judy.
An example would be a bit counting instruction.  Judy needs to
count bits in a word somewhat often -- but not such that it kills you
if you don't have a bit counting instruction.

Doug Baskins <doug@sourcejudy.com>