From: Robert Klemme <shortcutter@...>
Date: 2012-09-28T01:56:40+09:00
Subject: Re: Array Creation

On Wed, Sep 26, 2012 at 11:38 PM, Dave Aronson
<rubytalk2dave@davearonson.com> wrote:
> On Wed, Sep 26, 2012 at 3:47 PM, Tony Arcieri <tony.arcieri@gmail.com> wrote:
>
>> You're probably doing an inaccurate comparison. You're asking Ruby to
>> allocate an array with 614,400,000 slots, which in C is equivalent to the
>> same number of pointers, which depending on whether you're on a 32-bit or
>> 64-bit host translates to either 2.4GB or 4.9GB.
>
> It's worse than that.  ("He's dead, Jim!")  In Ruby it's not just a
> straightforward memory allocation.  The Array object itself needs
> additional setup, and for all I know there may be setup overhead for
> each individual slot.  A closer (but still not quite direct)
> comparison would be to allocate memory for a huge string, such as:
>
>   str = 'x' * 614400000
>
> Depending on your character encoding settings, this might allocate
> anywhere from 0.6G to 2.4G; you can force it higher by using some
> obscure 32-bit character, or of course a longer initial string or
> higher number.

It's yet even worse ("Aye, the haggis is in the fire for sure."):
after the malloc you get empty memory pages which do not exist.  The
OS just has to do some minor bookkeeping to remember all the reserved
address space of the process.  Only after that memory is actually
accessed memory pages need to be provided.  Since in absence of a
second parameter to Array.new all the Ruby Array slots need to be
initialized with 4 (Qnil) and cannot be left at 0 (Qfalse) all pages
need to be actually provided by the OS so they can be written.  With
the large number of pages a lot of them are likely swapped out to disk
which is awfully slooooouw.  See:

http://rxr.whitequark.org/mri/source/include/ruby/ruby.h#360

Cheers

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/