From: Martin Pirker <nospam@...>
Date: 2003-12-09T15:12:03+09:00
Subject: Re: handling large data sets

Ara.T.Howard <ahoward@ngdc.noaa.gov> wrote:
> On 8 Dec 2003, Martin Pirker wrote:
>> short version: What's your prefered way to handle large data sets used by
>> Ruby scripts?
> 
> rbtree (red-black) tree.
> 
> is has an interface like a hash but is _always_ sorted.  the sort method is
> determined by the keys '<=>' method.  it has also allows lower_bound and
> upper_bound searches.  it marshals to disk quite fast.  log(n) for insert,
> delete, and search.

very interesting
a way to hold a large working set of data, ordered and fast ops

looking at the README of rbtree, the extra methods "lower_bound,
upper_bound" (and others?) provided by rbtree don't seem to be
mentioned - is there a docu explaining them or guess from the test code?

 
> simply scanning memory is fast.

with numerical values I agree

however with mixed data taking notes of previous iteratings "feels"
cheaper to me

this approaches the border of "fit data to language data structures" or
"structure data best matched to problem domain"
I'm torn
as rather Ruby newbie I'm still in the transition to Ruby thinking, so
for me it's rather tapping in the dark and banging the head sometimes ;-)


thanks for your suggestions 

Martin