From: Geert Fannes <Geert.Fannes@...>
Date: 2006-03-14T06:46:27+09:00
Subject: Re: how to introduce reference objects into ruby

------_=_NextPart_001_01C646E7.8A45C4A4
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

>Well, consider that in a perfect world with zero storage overhead, =
you'd=20
>be looking at 4 bytes for the row, 4 bytes for the column, 8 bytes for =
a=20
>float, multiply by 800M elements active in your sparse array and by my=20
>calculations you've got 12GB of data before you even start hashing =
anything.

I use floats instead of doubles (only 4 byte), I end up a little above =
12Gig for storing the whole object. Next to this, I use ordered binary =
trees implemented in array, which have nice access  timings, small =
memory footprint, but are less flexible than AVL trees.

>Given that size of problem, I'd be strongly inclined to move it out of=20
>RAM into a relational database. Especially if you want to be able to=20
>iterate along any row or column, as you apparently do. In which case,=20
>you might as well use Ruby :-)

I would say a database is nice, but I fear it will be slow, disc access =
cannot be compared with memory access. It would probably be fast enough =
once I exactly know what operations I need to perform, but in the =
experimentation phase I am in now, I want to be able to try out things =
in a flexible and fast manner.

But to be honest, I never tried it, but maybe a nice Ruby class that =
hides all DB operations necessary would also work adequately. I'm just =
more experienced with data structures and memory allocations than with =
data bases. What data base system would you suggest? I guess a suitable =
one should have fast indexes and sorting routines. Each RDBMS can do =
this, but I think hude speed differences exist.

Greetings,
Geert.

------_=_NextPart_001_01C646E7.8A45C4A4--