From: Blackie <mudsweatandears@...>
Date: 2007-10-11T22:15:04+09:00
Subject: Re: Should *most* memory be release back to the system?

I do appreciate your help (and no that link is lost to the mists of
time as far as I can google.) :)

I do understand that they are not useful for leak detection, but this
is just observing the peak during the life of the script. During
"other_task" I can see the usage rise *and fall* so I know the OS is
reclaiming *some* memory in places...but the part I don't understand
is why it isn't returning to "tasks" original base of 50M when only
"task" is running.

I hate to sound dense, but I need to convince some fairly hard-headed
sysadmins.

On Oct 11, 8:40 am, Yohanes Santoso <ysantoso-rubyt...@dessyku.is-a-
geek.org> wrote:
> Blackie <mudsweatande...@gmail.com> writes:
> > If anyone can explain this I would appreciate it.
>
> It's your OS (I meant kernel and libc).
>
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/160726
> andhttp://www.crowdedweb.com/articles/2006/03/22/ruby-on-rails-and-appli....
>
> The second link is dead, but there may be cached copies around.
>
> The first one shows that on some OS, if there is a hole in your memory
> usage, that hole will not be returned by your libc to the kernel or
> the kernel won't reclaim it. Either way, your OS is not taking it
> back.
>
> The second one (if you ever find a copy) adds some more observations
> and also a handy tweak for FreeBSD that cause the OS to take back
> holes (IIRC).
>
> But most importantly, one should know that RSS and VSZ are not
> Accurate Measure of Memory Usage.
>
> YS.
>
>
>
> > I have an IT group knocking Ruby saying that it never releases memory
> > back to the system (to be available to procs other than Ruby) so
> > feeling somewhat defensive I went and wrote a dumb script that goes
> > back and forth between two methods, one cat'ing huge strings and the
> > other parsing an xml doc with Hpricot.
>
> > "task" tops off (in top) around 50M and "other_task" peaks around 150M
> > (on my machine, CentOS, lastest stable 1.8.6) but when we return to
> > running "task" for extended periods, memory usage remains at ~150M.
>
> > Forgive my ignorance. Can anyone explain this behavior or at least
> > point me to a place to educate myself?
>
> > Many thanks, the script I'm running is below.
>
> > --------
> > #!/usr/bin/env ruby
> > require 'rubygems'
> > require 'hpricot'
>
> > def other_task
> >   a = []
> >   9999.times  do
> >     a << "12345678901234567890123456789012345678901234567890" * 100
> >   end
> >   nil
> > end
>
> > def task
> >   # 500K xml data
> >   data = File.readlines("very_large_output.xml").to_s
> >   temp = Hpricot.XML data
> >   nil
> > end
>
> > puts "In task"
> > 10.times {|i| task; p i}
> > puts "In other task"
> > 100.times {|i| other_task; p i}
> > puts "In task (Should memory go down?)"
> > 100.times {|i| task; p i}