Re: Ruby on Neko ?

On Sunday 20 November 2005 04:38, Nicolas Cannasse wrote:
> Looking at Ruby implementation, every object needs to allocate a fairly
> big hashtable (11 buckets by default) in order to get a real O(1) access
> in practice - which can be O(n) in the worst case - and that cost (a)
> memory and (b) GC cycles when scanning it.
>
> When Neko for example is running into Apache, that's several hundreds of
> process having all theses living objects so in that case you're more
> often memory-bound than CPU-bound, it then makes sense to trade CPU for
> memory.

I have to agree that there are more considerations than just access speed.  
I'm running into this in REXML, where people are complaining that loading any 
document of a couple of megs will exhaust their memory, and even 1GB systems 
choke on documents greater than 5MB[1].  Part of the problem (perhaps the 
dominant problem) is that REXML Element nodes use hashtables for Attribute 
lists; so every Element has a hashtable[2].

A program being too slow is certainly a problem, but I'd argue that it isn't 
as big of a problem as having the program run your machine out of memory and 
then fail terribly.

Anyway, this doesn't meaningfully contribute to the discussion, but the thread 
is apropos to a problem I'm struggling with at the moment.



[1] This isn't a Ruby problem, of course.  This is programmer error, in 
failing to anticipate the large data sets.  My XML problem domains are always 
small, so, of course, are everyone else's :-)

[2] The problem in REXML is actually worse; I use Ruby hashes all over the 
place, with little regard, and now I'm having to go back and strip them out 
and replace them with balanced trees.  One thing I'm anticipating is having 
REXML choose between hash and red-black trees (or whatever I settle on) 
intelligently if it can determine the size of the document, and use trees if 
it can't.  However, it is more critical to make sure that REXML doesn't choke 
on large data sets.

-- 
--- SER

"As democracy is perfected, the office of president represents, 
more and more closely, the inner soul of the people.  On some 
great and glorious day the plain folks of the land will reach 
their heart's desire at last and the White House will be adorned 
by a downright moron."        -  H.L. Mencken (1880 - 1956)