From: ara.t.howard@...
Date: 2007-03-05T04:26:10+09:00
Subject: Re: [OT] Design question - mmap, insert/delete editing on huge binary files?

On Mon, 5 Mar 2007, Bill Kelly wrote:

> Hi,
>
> I'd like to create Ruby bindings for a library that may or
> may not exist yet.
>
> What I'd like to be able to do, is make an arbitrary number
> of insert/delete (cut/paste) style edits on huge binary
> files (> 4 GB).  There should be minimal overhead for initially "opening" the 
> file, and the inserts and deletes should be very fast, until the changes are 
> finally
> committed.
>

i do exactly this using guy's ruby mmap bindings all the time, check out the
api - it might already do much of what you want to do.  one issue you may have
is addressing - since c ints map to ruby fixnums i'm not sure you'll be able
to map files quite this big with the pure ruby impl (not that it couldn't be
fixed and guy may already have fixed it).  anyhow, you could almost certainly
overcome this by mapping using length offset.  i otherwords, map 4 chunks of
1gb.

  http://moulon.inra.fr/ruby/mmap.html

btw.  it's fast, look at the last example here

   http://codeforpeople.com/lib/ruby/nmap/nmap-0.1.0/README

regards.

-a
-- 
be kind whenever possible... it is always possible.
- the dalai lama