From: Joel VanderWerf <vjoel@...>
Date: 2006-08-15T06:15:56+09:00
Subject: Re: Looking for a Fast Persistent Store

Bob Hutchison wrote:
> Hi Ara,
> 
> On Aug 9, 2006, at 10:08 AM, ara.t.howard@noaa.gov wrote:
> 
>> On Wed, 9 Aug 2006, Bob Hutchison wrote:
>>
>>> Hi,
>>>
>>> I'm looking for a persistent store, where:
>>> * I can use it from Ruby
>>> * it is fast
>>> * it is transactional
>>> * it can update more than one store in a single transaction
>>> * simple String -> String mappings in the store are sufficient
>>> * ideally uses files in the filesystem
>>>
>>> These kinds of stores are normally implemented as persistent hashes or
>>> BTrees (or both). I know that Sleepycat's Berkeley DB
>>> <http://www.sleepycat.com/> does this, and I've used Java based 
>>> systems that
>>> do this. I also know of some C based things but they don't have Ruby
>>> wrappers. I can't find anything in the Ruby world, and I don't care 
>>> too much
>>> if it isn't pure Ruby.
>>>
>>> I know about Purple <http://purple.rubyforge.org/> and the QDMB Ruby 
>>> wrapper <http://qdbm.sourceforge.net/>. Neither do the multiple 
>>> hash/BTree in a transaction thing.
>>>
>>> The trick appears to be with transactions.
>>
>> check out joel's fsdb - it's very nice if you want to go pure ruby.  
>> i've used
>> it on several projects.
> 
> I'm already using that (version 0.5 -- I can't get to RAA right now for 
> some reason and there are no files on Rubyforge for fsdb so I don't know 
> if there is a more recent version). In version 0.5 the transactions were 
> not sufficient I think (it would be nice if I was wrong).

I apologize for the lack of fsdb stuff on the RubyForge page. I've never
found a comfortable way to automate gem uploads, so I still use my old
scripts for building this page:

http://redshift.sourceforge.net/

A quick scan there shows that fsdb-0.5 is the latest.

It's on my list to figure out how to automate the RubyForge dance (and I
think Ara or someone did this a while ago, so maybe it's a solved
problem now).

Now, on to your question...

Transactions in fsdb can be nested (and the transactions can be on
different dbs)--is that sufficient? This may not be what you mean by 
"single transaction", though.

But, like PStore, FSDB isn't very fast--it's pure ruby, it marshals
objects (by default--but you _can_ tell fsdb to write the value strings
directly rather than via marshal), and it pays the cost of thread- and
process-safety. One advantage over pstore is finer granularity.

Anyway, I hope your search is fruitful, whether it lands at bdb or 
sqlite or ...

-- 
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407