From: Charles Hixson <charleshixsn@...>
Date: 2012-11-13T06:12:17+09:00
Subject: Re: Extending Array instances

Brian Candler wrote:
> Charles Hixson wrote in post #1084111:
>> I'm trying to figure out a good way to extend an Array, when the items
>> may not be added in index order, and I don't know the final highest
>> index.  Does anyone have any suggestions?
> This happens automatically.
>
>>> a = [:foo, :bar]
> =>  [:foo, :bar]
>>> a[10] = :baz
> =>  :baz
>>> a
> =>  [:foo, :bar, nil, nil, nil, nil, nil, nil, nil, nil, :baz]
>
> As others have said: using a Hash may be more efficient, if there are
> large gaps - but then you may have to sort the keys if you want to
> iterate over it in order. It depends on your use case.
>
Hashes are slow compared to direct indexing.  Push might be a good 
answer, but not if it allocates storage item by item.  Allocation of 
memory is another expensive operation.  But I *am* considering using 
gdbm instead of an Array.  That is slow like a hash, but it eliminates 
needing to persist at a later time, and it conserves RAM usage.  OTOH, 
it would be a lot faster to just loop through the array marshaling out 
item by item.  Or perhaps just doing the entire thing in one blurt.

I'm expecting this array to eventually use over a GB of storage, so 
efficiency considerations are somewhat significant.  gdbm solves some of 
these, by not dealing with the entire array at once, but only with the 
recently active instances.  But it's slower, which I was trying to avoid 
by using an Array.  However, I don't know just how many items the Array 
will need to hold, it could easily be in the millions, and will at least 
be in the hundreds of thousands.  So pre-allocating isn't very 
desirable, but neither is adding instance by instance.  (Besides, they 
won't necessarily come in order.  It could well be that I'll get the 
100,000th entry immediately after getting the 21st.  No problem for a 
hash, but if I'm going to slow down to hash speed, I should probably 
just go directly to gdbm.)