From: Joel VanderWerf <vjoel@...>
Date: 2007-03-12T04:26:03+09:00
Subject: Re: You thoughts/philosphies on manual garbage collection

ara.t.howard@noaa.gov wrote:
> On Fri, 9 Mar 2007, dkmd_nielsen wrote:
> 
>> The process that initiated my message earlier (about deleting array
>> elements) is a rather long running process of rebuilding and
>> reconfiguring parameter files.  There hundreds of files, each with as
>> many as 22,000 parameters to processed.  For example, four small test
>> files ran in about two minutes.  There is a ton of string manipulation
>> going on, which probably translated into lots of trailing string parts
>> and pointer laying around RAM...clogging it up.  I was thinking of
>> manually initiating garbage collection after every five or ten files
>> processed.  Is that a smart thing?
>>
>> What are yours thoughts on manually initiated garbage collection?
>> What kinds of practices result in bits and pieces of objects and
>> pointers being left laying around in the ether of RAM?  Are there
>> tools that help see what happens to RAM while a process runs, like a
>> debugger does with variables?
>>
>> Thanks for everything
>> dvn
> 
> if you can fork - that's the best - then you just let each child's death 
> clean
> up that sub-segment of work's memory.

One caution: mark-and-sweep GC and fork don't always play well together, 
in terms of sharing memory pages. The mark algorithm needs to touch all 
live objects in the heap. The child inherits the parent's heap, with 
copy on write. If the parent has a large heap, and the child does a GC, 
  all those pages are copied into the child's address space. Memory 
usage will scale badly as the number of child processes grows. (Perhaps 
you factor your process into one child for each of the hundreds of files?)

It can be a good idea to GC.disable in the child, in some cases:

- parent has large heap, and

- child lifespan and allocation rate are such that is does not need to GC

Some benchmarks:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/186561

-- 
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407