From: Robert Klemme <shortcutter@...>
Date: 2010-10-29T22:54:54+09:00
Subject: Re: Ruby 1.9.1 linear memory increase in a simple for loop.

On Fri, Oct 29, 2010 at 3:17 PM, Ritwik Banerjee
<ritwik.banerjee@gmail.com> wrote:
> The following text code shows a linear rise in memory usage. Since this
> is a simple loop, I am guessing there's something fundamentally wrong
> (either in my understanding of ruby's garbage collection, or in ruby's
> garbage collection):
>
> memtest.rb
> ----------
> require 'matrix'
> a = [[1,0,1,0,0,1,1,1],[0, 0, 0, 0, 0, 1, 1, 0],[1, 1, 1, 1, 0, 1, 1,
> 1],[0, 0, 1, 1, 1, 1, 0, 1],[0, 0, 1, 1, 1, 1, 0, 1],[0, 0, 0, 1, 1, 1,
> 1, 1],[0, 1, 1, 1, 1, 1, 1, 1],[0, 1, 0, 1, 1, 0, 1, 1]]
>
> m = Matrix.rows(a)
> n = Matrix.identity(8)
>
> for i in 1..100 do
>    n = n * m;
>    physical_mem_usage = `ps -o rss= -p #{Process.pid}`.to_i;
>    virtual_mem_usage = `ps -o vsz= -p #{Process.pid}`.to_i;
>    puts "#{i}: VMU = #{virtual_mem}; PMU = #{physical_mem}."
>    puts ObjectSpace.count_objects()
> #   ObjectSpace.garbage_collect
>    STDOUT.flush
> end
>
> ---------
>
> This code shows a linear increase in memory usage. The
> ObjectSpace.count_objects call shows a steady linear increase in counts
> of T_ARRAY, T_HASH, T_STRUCT, T_STRING and T_OBJECT.
>
> Can anyone explain this? Typically memory explosions occur in complex
> codes with lots of references, and in those cases the memory rise is
> exponential. In the above loop, invoking garbage collection manually
> (it's commented out in the snippet above) slows the memory increase, but
> only by a constant factor.
> For example, if the loop used up 30 kB in 100 iterations originally, it
> uses up 30kB in 500 iterations after invoking garbage collection
> manually.
>
> Understanding this is critical for me since I am dealing with matrices
> of enormous sizes, so even a small memory leak can jeopardize my
> project.

AFAIK there is a certain memory threshold below which the interpreter
never does GC.  This helps make small scripts run fast.  You probably
need to let your program run much longer and monitor memory to see the
long term effect.  Only if you see dramatic increase in the long run
you may conclude that you found a leak IMHO.

Kind regards

robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/