From: Matthew Bloch Date: 2002-07-13T02:52:27+09:00 Subject: Re: Wiring up the Boehm GC to the Ruby interpreter On Friday 12 July 2002 17:05, you wrote: > Looks like what you've done is to replace ruby's calls to malloc with a > garbage collected version. Now you've got two garbage collectors > running. No, I took out the calls to the Ruby GC. There's obviously still some Ruby-GC overhead associated with each object, and probably a little explicit marking still going on, possibly even enough to add further delay, which I'll investigate more thoroughly, but it's definitely not running the old mark & sweep code on a regular basis because it's not compiled in. > To really replace ruby's garbage collector you should change gc.c to > match your tastes. The Boehm collector uses essentially the same > algorithm already used by ruby's garbage collector. The basic algorithm is the same, as I understand it, but it has an incremental mode, which is what I wanted it for. > A good place to start might be to disable garbage collection during the > critical areas of your program via GC#disable and then call > GC#garbage_collect only when you've got plenty of time - maybe only once > in a blue moon - maybe never. Doesn't work unfortunately :-/ I tried this during some animations but even calling GC.start after enabling the GC caused the memory gained during the 'critical' section to be recovered. Eventually it just runs out of memory, so calling GC.disable for any length of time seems to be an unacceptable risk (at least it is for a game running on a public terminal that's meant to take people's money). I had a reply from Hans Boehm on the gclist which was Cc'd here but won't have got here because he's not a subscriber: ------------------------------------------------------------------------------ RE: [gclist] Wiring up the Boehm GC to the Ruby interpreter From: "Boehm, Hans" To: "'Matthew Bloch'" , ruby-core@ruby-lang.org Cc: gclist@lists.iecc.com Date: Fri, 12 Jul 2002 10:09:32 -0700 Some comments/questions from someone who knows the collector but not Ruby: 1) The top priority here is identifying the cause of the leak. Once there is a leak, the collector will have to do a lot more work tracing things that should be garbage, pause times will increase, etc. Try building the collector with KEEP_BACK_PTRS and DBG_HDRS_ALL (or with --enable-full-debug), build the application with GC_DEBUG defined before gc.h is included, and so that it calls the all-caps versions (e.g. GC_MALLOC) of the GC_ routines, run your application for long enough that the heap has grown substantially as a result of the leak (say by a factor of 4), and the call GC_generate_random_backtrace() several times. This should tell you why some typical objects are being retained. If you waited long enough, the probability is high that the "random" object is one that was leaked. When I looked at some corresponding issue with Python a long time ago, a big part of the problem seemed to be a few optimizations in the runtime that "knew about" Python's original memory management. Any code that keeps its own free lists should be looked upon with some suspicion, since those free lists will be treated as live by the collector. If those free lists point to other known garbage, that will also be retained. But it's also possible that the collector will need some type information in a few places, e.g. when you're allocating arrays containing image data. 2) Did you turn on incremental collection with GC_enable_incremental()? Otherwise the GC_collect_a_little() calls will have no effect, other than possibly triggering a complete GC at that point. On a Linux system (and on many others) turning on incremental collection will cause the heap to be write protected. The collector will handle the resulting write faults, unless they occur inside a system call. Thus you will need to be careful with, and preferably avoid, system calls that write directly to the garbage-collected heap. (If the application is single-threaded, or nearly single-threaded, you can also wrap the system calls with some routines provided by the collector. In some cases, it's also OK to have system calls write to objects allocated with GC_MALLOC_ATOMIC. But this has turned out to be enough of an issue that I believe there are relatively few clients for the incremental collector.) Hans ------------------------------------------------------------------------------ Anyhow I'll take another look at this with the above in mind-- I'm keen to see these delays gone from Ruby's GC for good. -- Matthew Bloch Bytemark Computer Consulting http://www.bytemark.co.uk/ tel. +44 (0) 8707 455026