From: zdennis <zdennis@...>
Date: 2006-02-27T07:47:00+09:00
Subject: Re: Memory Question

Eric Hodel wrote:
> Ruby does not feature a compacting garbage collector so you may  simply 
> have a lot of nearly-empty pages of memory that cannot be  returned to 
> the operating system.
> 
> If you still suspect a leak I suggest you first see if you can  achieve 
> the same results by simply generating objects without any DB  and also 
> with just the mysql driver, no Rails.
> 

Well I did some digging, and I'm left with more questions. The following script does NOT collect the 
800,000 strings created (this doesn't use Mysql, Rails or anything else, just ruby core). I ran this 
in accordance with a couple other scripts which utilize 150mb+ of memory, to see if the os would 
somehow force ruby to do better GC'ing, it didn't and swap was used. On my system this script 
consistently used 42Mb of memory, and the string count never went below 800,000. Why would the 
strings not get gc'd when they are out of scope ?

----------test1.rb----------------
def count_objects_for clazz
   c = 0
   ObjectSpace.each_object{ |o| c+=1 if o.is_a? clazz }
   c
end

class A
  def run
    arr = []
    800000.times { arr << "d" * 7 }

    puts "check now"
    sleep 10
   end
end

puts count_objects_for( String )
A.new.run
puts count_objects_for( String )
GC.start
puts count_objects_for( String )

Thread.new do
   loop do
     puts count_objects_for( String )
     GC.start
     sleep 5
   end
end.join
-------end test1.rb-------------


Send test, same script as above but slight modification to call Array#clear. Now this actually 
collects the Strings. I am specifically removing reference in the below code by clearing the Array 
arr. But in the above code I don't reference arr or it's value anywhere outside of the method it is 
in, so after that call occurs shouldn't it be fair game for the garbage collector?

-------- test2.rb-----------
def count_objects_for clazz
   c = 0
   ObjectSpace.each_object{ |o| c+=1 if o.is_a? clazz }
   c
end

class A
  def run
    arr = []
    800000.times { arr << "d" * 7 }
    puts "check now"
    sleep 10
    arr.clear
   end
end

puts count_objects_for( String )
A.new.run
puts count_objects_for( String )
GC.start
puts count_objects_for( String )

Thread.new do
   loop do
     puts count_objects_for( String )
     GC.start
     sleep 5
   end
end.join
-------end test1.rb-------------

I modified this second script to run with 8 million string also. When the Strings are GC'd the ruby 
processes memory usage goes down, but not as much as I'd think. With 8 million strings I am getting 
330+Mb of memory in use, but when I GC them, memory only seems to go 191Mb, where I would think it 
would fall back down in the single/double digit's, perhaps 15M or 20M. Any reason why ruby doesn't 
let go? Is this a problem? Right now I have ran this with ruby 1.8.3 (2005-06-23) [i486-linux], and 
ruby 1.8.4 (2005-12-24) [powerpc-linux] with similar results.

Zach