From: Robert Klemme <shortcutter@...>
Date: 2007-01-17T00:15:10+09:00
Subject: Re: efficient access to a large int32 array

On 16.01.2007 15:29, Martin DeMello wrote:
> On 1/16/07, Robert Klemme <shortcutter@googlemail.com> wrote:
>> On 16.01.2007 15:10, Martin DeMello wrote:
>> > I have a file that is a dump of a large array of 32-bit unsigned ints
>> > (bitvectors actually). All I need is to load this into memory and have
>> > fast, efficient access to the array members, and a few bitmasking
>> > operations - the array will never be written to, and ideally should
>> > not need to be bounds-checked. Should I be looking at something like
>> > mmap or narray to do this, or would I be better served by a custom C
>> > extension?
>>
>> How large is "large"?
> 
> Order of 1 MB - not gigantic, but not trivial either.

Um, I would not say 1MB is large.  What accesses do you do?  Do you just 
access based on index?  Do you often need to load this data? etc.

In your case I'd start out writing a Ruby version (either using a single 
String or an Array with integers) and change that into an extension if 
performance is insufficient.  My 0.02 EUR...

Kind regards

	robert