From: Eric Wong <normalperson@...>
Date: 2014-05-27T03:21:29+00:00
Subject: [ruby-core:62770] Re: [ruby-trunk - Feature #9638] [PATCH] limit IDs to 32-bits on 64-bit systems

I am OK with closing this issue (but I'm not sure if I have permissions
to close on redmine).

However, your applications need more than 2**32 different symbols?
That scares me :*(
How much memory do your Ruby processes use?

The Symbol table currently takes at least (48 + 48 + 40 = 136) bytes per
symbol on 64-bit, so 136 * (2 ** 32) is 544 gigabytes just for the
symbol table (w/fstrings) in your app.  That does not even account for
memory of symbols with string representations longer than 23 bytes,
nor the memory for hash table buckets.

I need to know because I am also looking into using khash[1] for the
symbol table.  By default, khash internal buckets/counters are all
32-bits.  We can tweak khash to use 64-bit counters if needed,
but 2**32 symbols really should be enough.

The symbol table with khash might reduce memory overhead to ~90 bytes
per-symbol on average, though...

[1] git clone https://github.com/attractivechaos/klib.git
    mruby also uses khash for (all?) its hash table needs.