From: Daniel Berger <Daniel.Berger@...>
Date: 2005-08-18T00:19:09+09:00
Subject: Memory allocation, Unix vs Windows (was: Ruby on Unix vs. Windows)

Lothar Scholz wrote:
> Hello Daniel,
> 
> 
>>>The analysis builds a profile in memory of over 3.6 GB of 
>>>data on disk.  On the Solaris box, it takes about 35 mins and 
>>>uses about 700 MB of RAM.  It would not complete on the 
>>>windows box using the full data set, bombing with "failed to 
>>>allocate memory (NoMemoryError)".  There was nearly 800 MB of 
>>>RAM free on the windows box as well as having a 4 Gig swap available.
>>>
>>>Is windows that inefficient with memory allocation or is this 
>>>a ruby implementation issue on windows?
> 
> 
> BD> I suspect mostly the latter, with a little bit of the former mixed in
> BD> for good measure.  I'm pretty sure the GC algorithm is different on
> BD> Windows, though I can't remember the details now.
> 
> No its exactly the same. Remember that the ruby GC has an extremely
> dumb implementation.

Well, consider the following script.  On Solaris the memory peaks at 
about 8k, but never drops.  On Windows it climbs to about 11k, then 
drops, climbs again, drops, etc.

I could have sworn I read somewhere that it had to do with the way Ruby 
was allocating memory.  Or is it just the operating system?

Regards

Dan

# blocktest.rb
class BlockTest
    def test
       yield "hello" * 1000
    end
end

if $0 == __FILE__
    require "sys/proctable"
    include Sys

    p Process.pid

    # Windows and Solaris
    if File::ALT_SEPARATOR
       mem = ProcTable.ps(Process.pid).working_set_size
       vmem = ProcTable.ps(Process.pid).page_file_usage
    else
       mem = ProcTable.ps(Process.pid).size
       vmem = ProcTable.ps(Process.pid).rss
    end

    puts "Starting Memory: #{mem}"
    puts "Starting VMemory: #{vmem}"

    b = BlockTest.new

    100000.times{ |n|
       b.test{ }
       sleep 0.5

       if File::ALT_SEPARATOR
          cur_mem = ProcTable.ps(Process.pid).working_set_size
          cur_vmem = ProcTable.ps(Process.pid).page_file_usage
       else
          cur_mem = ProcTable.ps(Process.pid).size
          cur_vmem = ProcTable.ps(Process.pid).rss
       end

       if cur_mem < mem
          puts "Memory dropped to #{cur_mem} from #{mem} on iter #{n}"
          puts "Current VMemory: #{cur_vmem}"
          puts
          mem = cur_mem
       else
          mem = cur_mem
          vmem = cur_vmem
          if n % 100 == 0
             puts "Current Memory: #{mem}"
             puts "Current VMemory: #{vmem}"
             puts
          end
       end

       if cur_vmem < vmem
          puts "VMemory dropped to #{cur_vmem} from #{vmem} on iter #{n}"
          puts "Current Memory: #{cur_mem}"
          puts
          vmem = cur_vmem
       else
          mem = cur_mem
          vmem = cur_vmem

          # Print every 100th iteration
          if n % 100 == 0
             puts "Current Memory: #{mem}"
             puts "Current VMemory: #{vmem}"
             puts
          end
       end
    }
end