Re: how to introduce reference objects into ruby

>Well, consider that in a perfect world with zero storage overhead, you'd 
>be looking at 4 bytes for the row, 4 bytes for the column, 8 bytes for a 
>float, multiply by 800M elements active in your sparse array and by my 
>calculations you've got 12GB of data before you even start hashing anything.

I use floats instead of doubles (only 4 byte), I end up a little above 12Gig for storing the whole object. Next to this, I use ordered binary trees implemented in array, which have nice access  timings, small memory footprint, but are less flexible than AVL trees.

>Given that size of problem, I'd be strongly inclined to move it out of 
>RAM into a relational database. Especially if you want to be able to 
>iterate along any row or column, as you apparently do. In which case, 
>you might as well use Ruby :-)

I would say a database is nice, but I fear it will be slow, disc access cannot be compared with memory access. It would probably be fast enough once I exactly know what operations I need to perform, but in the experimentation phase I am in now, I want to be able to try out things in a flexible and fast manner.

But to be honest, I never tried it, but maybe a nice Ruby class that hides all DB operations necessary would also work adequately. I'm just more experienced with data structures and memory allocations than with data bases. What data base system would you suggest? I guess a suitable one should have fast indexes and sorting routines. Each RDBMS can do this, but I think hude speed differences exist.

Greetings,
Geert.