Re: Simple Ruby DB apps/programs ...

Matt Armstrong wrote:
> "Useko Netsumi" <[e-mail]> writes:
> 
> 
>>Thanks and please forgive my lack of knowledge of pstore.
>>
>>
>>>pstore.rb uses File.flock to lock the pstore database.
>>
>>So, does this mean that it can ONLY serve one users for add/update
>>record(s) in the DB and mutli-user for read/search?
> 
> 
> Yes.
> 
> 
> 
>>Is pstore is widely use? Thanks
> 
> 
> The old RAA used a pstore to store the data for every ruby project
> listed (maybe the new one does too, I don't know).  It works fine so
> long as your data set is not large -- it reads the *whole* data set
> into memory and writes it all back out each time it is opened/closed.
> 

These two problems are part of why I wrote fsdb (on RAA, and at 
http://redshift.sourceforge.net/fsdb/). The basic idea is that you 
select a directory to be your "database", and FSDB gives you a db object 
that has a database interface and accessed that dir and it subdirs:

db = FSDB::Database.new('/tmp')
db['foo/bar'] = {:x => 1}
db.edit 'foo/bar' do |bar|
   bar[:x] += 1      # inside transaction
end
db['foo/bar'][:x]   # ==> 2

FSDB makes life easy if you have a pre-existing file structure with 
various formats, and you want to treat them as a database. You can 
define your own mapping from path strings to file formats (e.g., ".txt" 
files are read as strings, ".obj" as marshalled objects, etc.; 
naturally, you can use regexes.). Also, FSDB is thread-safe and 
process-safe.

A quick comparison with PStore:

Like pstore, fsdb is:

- pure ruby, cross-platform (well, not quite there on windows yet--see 
below)

- ruby license

- simple API

- simple implementation (though less so)

- transaction based, with abort capability

However, pstore has some drawbacks:

- not thread-safe (though it is process-safe, as far as flock goes)

- coarse granularity (transaction applies to whole pstore), does not 
scale well

- uses marshalling, even if your data is is some other format that 
doesn't require the overhead of marshalling (e.g., strings, binary data)

Other advantages of fsdb:

- can use YAML (or other formats) where you want to (e.g., in user 
settings, but perhaps not for large ruby objects or binary data). It's 
easy to specify a mapping from path patterns to data formats.

- thread-safe, so you can easily wire it up to a multithreaded server, 
like drb (see examples/server.rb)

- objects in the database are just files, and so can be manipulated with 
standard command line tools (including rsync, versioning tools, etc.).

- granularity is up to you: you can have lots of little objects, if you 
want (there are tradeoffs, of course)

- scales as well as the fs does

- thoroughly stress-tested, partially unit-tested on linux and solaris. 
On windows, all examples, benchmarks, and tests except the stress test
(tests/test-concurrency.rb) run correctly with the ruby 1.8.0
single-click installer edition. (The stress test does run on windows
with ruby 1.8.1 built with mingw, so I'm itching to get /\ndy's 1.8.1.)

- Can select fcntl-based locking (if platform supports it), which should 
allow safe use with NFS mounts on linux

Disadvantages of fsdb:

- If you want indexes, you need to implement them on top of fsdb. You 
could do:

   db['name-index'] = RBTree.new

and define your own methods for adding/removing objects that also 
updates the name-index. The name-index maps names (or other keys) to 
paths in the db.

- Likewise, no SQL, out of the box.

- Efficiency and reliability depends on the fs. (My benchmarks get on 
the order of 1000 transactions per sec on a 3 yr old linux box. Windows 
seems a bit slower...)