From: braver <deliverable@...>
Date: 2007-05-27T06:05:02+09:00
Subject: querying persistent ruby objects in memory

I have a data-mining task which loads data as a big XML tree (10+ MB)
and then reorganizes it.  Even loading it with Hpricot takes 10-20
seconds.  I don't want to do it for every manilupation I want to try,
especially for sequences of transformations.

Thus I wonder what's a good way to keep the huge object in memory
between the runs of querying scripts.  Can Rails be used for that?
I'd rather avoid writing a client-server platform, or using it per se,
unless there's already an existing one.  A vague intuition is, it
should be something like threads -- one thread parses XML and keeps it
in memory, another starts up later, somehow joins the memory space of
the first one, queries/transforms it, and ends.  Then other queries/
transformations can all be run.  Is there anything like it?

Cheers,
Alexy