Re: Wiring up the Boehm GC to the Ruby interpreter

On Friday 12 July 2002 17:05, you wrote:
> Looks like what you've done is to replace ruby's calls to malloc with a
> garbage collected version.  Now you've got two garbage collectors
> running.

No, I took out the calls to the Ruby GC.  There's obviously still some 
Ruby-GC overhead associated with each object, and probably a little explicit 
marking still going on, possibly even enough to add further delay, which I'll 
investigate more thoroughly, but it's definitely not running the old mark & 
sweep code on a regular basis because it's not compiled in.

> To really replace ruby's garbage collector you should change gc.c to
> match your tastes.  The Boehm collector uses essentially the same
> algorithm already used by ruby's garbage collector.

The basic algorithm is the same, as I understand it, but it has an 
incremental mode, which is what I wanted it for.

> A good place to start might be to disable garbage collection during the
> critical areas of your program via GC#disable and then call
> GC#garbage_collect only when you've got plenty of time - maybe only once
> in a blue moon - maybe never.

Doesn't work unfortunately :-/  I tried this during some animations but even 
calling GC.start after enabling the GC caused the memory gained during the 
'critical' section to be recovered.  Eventually it just runs out of memory, 
so calling GC.disable for any length of time seems to be an unacceptable risk 
(at least it is for a game running on a public terminal that's meant to take 
people's money).

I had a reply from Hans Boehm on the gclist which was Cc'd here but won't 
have got here because he's not a subscriber:

------------------------------------------------------------------------------
RE: [gclist] Wiring up the Boehm GC to the Ruby interpreter
From: "Boehm, Hans" <[e-mail]>
To: "'Matthew Bloch'" <[e-mail]>, [e-mail]
Cc: [e-mail]
Date: Fri, 12 Jul 2002 10:09:32 -0700
 
Some comments/questions from someone who knows the collector but not Ruby:

1) The top priority here is identifying the cause of the leak.  Once there is 
a leak, the collector will have to do a lot more work tracing things that 
should be garbage, pause times will increase, etc.  Try building the 
collector with KEEP_BACK_PTRS and DBG_HDRS_ALL (or with --enable-full-debug), 
build the application with GC_DEBUG defined before gc.h is included, and so 
that it calls the all-caps versions (e.g. GC_MALLOC) of the GC_ routines, run 
your application for long enough that the heap has grown substantially as a 
result of the leak (say by a factor of 4), and the call 
GC_generate_random_backtrace() several times.  This should tell you why some 
typical objects are being retained.  If you waited long enough, the 
probability is high that the "random" object is one that was leaked.

When I looked at some corresponding issue with Python a long time ago, a big 
part of the problem seemed to be a few optimizations in the runtime that 
"knew about" Python's original memory management.  Any code that keeps its 
own free lists should be looked upon with some suspicion, since those free 
lists will be treated as live by the collector.  If those free lists point to 
other known garbage, that will also be retained.

But it's also possible that the collector will need some type information in 
a few places, e.g. when you're allocating arrays containing image data. 

2) Did you turn on incremental collection with GC_enable_incremental()?  
Otherwise the GC_collect_a_little() calls will have no effect, other than 
possibly triggering a complete GC at that point.  On a Linux system (and on 
many others) turning on incremental collection will cause the heap to be 
write protected.  The collector will handle the resulting write faults, 
unless they occur inside a system call.  Thus you will need to be careful 
with, and preferably avoid, system calls that write directly to the 
garbage-collected heap.  (If the application is single-threaded, or nearly 
single-threaded, you can also wrap the system calls with some routines 
provided by the collector.  In some cases, it's also OK to have system calls 
write to objects allocated with GC_MALLOC_ATOMIC.  But this has turned out to 
be enough of an issue that I believe there are relatively few clients for the 
incremental collector.)

Hans
------------------------------------------------------------------------------

Anyhow I'll take another look at this with the above in mind-- I'm keen to 
see these delays gone from Ruby's GC for good.

-- 
Matthew Bloch                 Bytemark Computer Consulting
                                http://www.bytemark.co.uk/
                                  tel. [phone]