From: Yohanes Santoso <ysantoso-rubytalk@...>
Date: 2005-10-16T13:51:34+09:00
Subject: Re: Help! define_method leaking procs...

Eric Mahurin <eric_mahurin@yahoo.com> writes:

> --- Yohanes Santoso <ysantoso-rubytalk@dessyku.is-a-geek.org>
> wrote:
>
>> Eric Mahurin <eric_mahurin@yahoo.com> writes:
>> 
>> > n=2**13;(1..n).each{|i|
>> >
>>
> a=(1..i).to_a;self.class.send(:define_method,:"f#{i}"){i*i}};GC.start;
>> >
>>
> IO.readlines("/proc/#{Process.pid}/status").grep(/VmSize/).display'
>> > VmSize:   172028 kB
>> 
>> > ruby -e '
>> > n=2**13;(1..n).each{|i|
>> >
>>
> a=(1..i).to_a;self.class.send(:define_method,:"f#{i}"){i*i};a=nil};GC.start;
>> >
>>
> IO.readlines("/proc/#{Process.pid}/status").grep(/VmSize/).display'
>> > VmSize:    11504 kB
>> 
>> Stop right there. I want to remind people that you can't use
>> VmSize as
>> a leak indicator. In some OS, VmSize is an always increasing
>> number. Memory allocated in a process is not returned to the
>> OS until
>> the process dies.
>
> OK.  Pick another way to measure memory.  top shows the same
> memory as above for me.  Or make n=2**15.  This brings my
> machine (768MB) to its knees:
>
> ruby -e 'n=2**15;(1..n).each{|i|
> a=(1..i).to_a;self.class.send(:define_method,:"f#{i}"){i*i}};GC.start;
> IO.readlines("/proc/#{Process.pid}/status").grep(/VmSize/).display'
>
> I'd estimate it would use about 3GB.  And if you put an a=nil
> after the define_method, you get this:
>
> ruby -e 'n=2**15;(1..n).each{|i|
> a=(1..i).to_a;self.class.send(:define_method,:"f#{i}"){i*i};a=nil};GC.start;
> IO.readlines("/proc/#{Process.pid}/status").grep(/VmSize/).display'
> VmSize:    31032 kB


That is still not a valid way to detect leak as that still depends on
VmSize value. Ruby could be freeing every allocation and the Vmsize
would still grow. 

I posted two versions because the first version was
1. has an off-by-1 error, 
2. does not show that the order of free() matters.

Try this diff where the hole is freed in different order. There is no
memory leak this time. The same number of allocations and frees, but
it has no memory leak!

Cheers,
YS.


--- leak.c	2005-10-16 00:18:34.000000000 -0400
+++ noleak.c	2005-10-16 00:40:36.000000000 -0400
@@ -74,6 +74,11 @@
       goto die;
     }
     hole[0] = '\0';
+    if (hole) {
+      fprintf(stderr, "Freeing hole\n");
+      free(hole);
+      hole=NULL;
+    }
   }
  die:
   fprintf(stderr, "unwinding\n");
Freeing immediately
----BEFORE----
malloc count = 0
free count   = 0
VmSize:	    1572 kB
VmLck:	       0 kB
VmRSS:	     364 kB
VmData:	     156 kB
VmStk:	      88 kB
VmExe:	       4 kB
VmLib:	    1280 kB
VmPTE:	      16 kB
Executing leak test
Allocating hole
Freeing hole
unwinding
----AFTER-----
malloc count = 8193
free count   = 8193
VmSize:	    1572 kB
VmLck:	       0 kB
VmRSS:	     440 kB
VmData:	     156 kB
VmStk:	      88 kB
VmExe:	       4 kB
VmLib:	    1280 kB
VmPTE:	      16 kB
Not freeing immediately
----BEFORE----
malloc count = 0
free count   = 0
VmSize:	    1572 kB
VmLck:	       0 kB
VmRSS:	     364 kB
VmData:	     156 kB
VmStk:	      88 kB
VmExe:	       4 kB
VmLib:	    1280 kB
VmPTE:	      16 kB
Executing leak test
Allocating hole
Freeing hole
unwinding
freeing 8192 elements
----AFTER-----
malloc count = 8193
free count   = 8193
VmSize:	    1572 kB
VmLck:	       0 kB
VmRSS:	     512 kB
VmData:	     156 kB
VmStk:	      88 kB
VmExe:	       4 kB
VmLib:	    1280 kB
VmPTE:	      16 kB