From: "M. Edward (Ed) Borasky" <znmeb@...>
Date: 2006-08-28T03:19:10+09:00
Subject: Re: Can Anyone Explain This Memory Leak?

Marshall T. Vandegrift wrote:
> As Ara described in another message in this thread, Unix-y heap implementations
> (including glibc's) typically get new process memory from the kernel by calling
> brk() to extend the process's data segment.  This can increase the apparent
> memory usage of code because the heap implementation can only move the end of
> the data segment backwards (releasing memory to the system) as far as the last
> page containing in-use memory.
Patient: Doctor, it hurts when I do that!
Doctor: OK ... don't do that!

Patient: I have this pain right here.
Doctor: Have you ever had this before?
Patient: No.
Doctor: Well, you have it now!

> Ara also mentioned in another message that Mutex is faster than Synchronize,
> and hypothesized that this was leading to a pathological interaction with the
> ruby garbage collector when creating as many threads as your example does.  My
> data suggest that the observed "Mutex memory leak" /is/ caused by Mutex's
> relative speed, but that it is due to an interaction with the system heap
> implementation rather than with ruby's garbage collector.  I haven't track down
> the exact factors at work, but I'd guess that 'mutex.rb' is (a) allocating new
> memory within the time window the heap implementation holds onto unused system
> memory, and (b) under some circumstances sufficiently fragments memory to force
> the heap implementation to allocate more system memory than required,
> eventually bringing the process under the gaze of the OOM killer.
> 
> I hope this helps.
> 
> [1] I used glibc's provided hooks for malloc/realloc/free and wrote a
>     quick-and-dirty custom trampoline for brk().  Yes, I know about ltrace, but
>     it only caught brk() as a system call, which slowed things down by a factor
>     of 1000 (!).  And yes Solaris DTrace would have made this easy, but I made
>     my graphs while waiting for the 2.5Gb Solaris Express DVD to finish
>     downloading. :-) I can provide my code if anyone wants it.

I'm more interested in what these tests do on Solaris :). We have
Windows, Linux, Mac OS and someone I think ran it on BSD. Just out of
curiosity, will these test cases run on YARV?