From: Jano Svitok <jan.svitok@...>
Date: 2008-02-22T20:38:17+09:00
Subject: Re: Newbie Mem Leak Issue

On Fri, Feb 22, 2008 at 11:35 AM, Jano Svitok <jan.svitok@gmail.com> wrote:
> On Fri, Feb 22, 2008 at 7:32 AM, Joel VanderWerf
>  <vjoel@path.berkeley.edu> wrote:
>
> > Joel VanderWerf wrote:
>  >  > Keith Barr wrote:
>  >  >> Interesting.  I am running (obviously) on Windows and I also have a
>  >  >> Linux box, both on patch 111, and both leaking.  It isn't a quick
>  >  >> leak, if you run for an a while it might show up.  How long did you
>  >  >> let it go?
>  >  >
>  >  > Only 10 cycles, but it didn't seem to be growing...
>  >
>  >  After 150 cycles, still under 18M. (This is the same code as yours with
>  >  the one dir path modification.)
>
>
>
> I have run it under winxp, the code and results can be found at
>  http://bryvecer.sk/leak/
>  It seems that it might slowly leak something, but I don't know for sure.
>
>  I have used pslist from sysinternals.com for memory measurements.

I have run Ruby MemoryValidator by SoftwareVerify on your code:

This version doesn't leak:

#DATA_PATH = "C:/Program Files" # 80_000 files
DATA_PATH = "C:/Program Files/Microsoft Visual Studio 8" # 19_000 files

class LeakTest
 def get_files(pathname)
   listing = Array.new

   Dir.glob("#{pathname}/*",0) do |f|
     # add the files
     listing << f
     # now add directories.
     if(File.directory?(f))
       returnlist = get_files(f)
       returnlist.each {|filename| listing << filename }
     end
   end
   return listing
 end

 include GC
 def cleanup
   GC.start
   sleep(5)
 end

 def monitor
   listing = get_files(DATA_PATH)
   sleep(5)
 end
end

lt = LeakTest.new
while true
 lt.monitor
 lt.cleanup
end

This is my version, that seems to use less objects
(I'm 1. using an accumulator, and 2. leave the arrays as are, until
they are all ready, and then flattening them,
so GC does it work once for all objects) It's possible that 1. without
2. is better.

#DATA_PATH = "C:/Program Files" # 80_000 files
DATA_PATH = "C:/Program Files/Microsoft Visual Studio 8" # 19_000 files

class LeakTest
 def get_files(listing, pathname)
   files = Dir.glob("#{pathname}/*", 0)
   # add the files
   listing << files
   # now add directories.
   files.each do |f|
     get_files(listing, f) if File.directory?(f)
   end
   return listing
 end

 include GC
 def cleanup
   GC.start
   sleep(5)
 end

 def monitor
   listing = []
   get_files(listing, DATA_PATH)
   listing.flatten!
   sleep(5)
 end
end

lt = LeakTest.new
while true
 lt.monitor
 lt.cleanup
end

It seems that the problem is in the little debug messages, that are
hard to collect.