From: "M. Edward (Ed) Borasky" <znmeb@...>
Date: 2007-08-12T12:50:02+09:00
Subject: Re: Interesting garbage collection article on LTU

Gregory Brown wrote:
> On 8/11/07, John Joyce <dangerwillrobinsondanger@gmail.com> wrote:
> 
>>> And Linux is notoriously proud of the mantra "free memory is wasted
>>> memory". It usually won't kick a page out to disk until it needs it
>>> for
>>> something else. So the resident set size of a Linux process/thread may
>>> have little or nothing to do with its actual working set.
>>>
>> Might it not have something to do with the liberal use of symbols in
>> Rails?
>> AFAIK symbols don't get GC'd, or do they?
> 
> It's true that symbols can create memory leaks[0], but the tendency of
> Rails to hog memory is mostly because its a big system that hasn't
> necessarily been designed for performance.  Creating a bunch of
> temporary objects without discarding them properly, using Array#shift
> in Ruby 1.8, or leaving references to objects open in closures without
> realizing it is more likely to make up a much more significant part of
> what causes processes to balloon.
> 
> Also keep in mind that Rails keeps quite a bit of objects alive just
> in normal use.  This is not something that will be easy to change.
> 
> That having been said, I'm sure there are places in the code base when
> something like:
> 
> something.to_s == "foo"
> 
> could be used instead of:
> 
> something.to_sym == :foo
> 
> This would not be my first place to look for memory optimizations in
> Rails though.
> 
> Saving my criticism for last, the OP as well as Ed didn't mention
> Rails, and Ruby != Rails. :)
> 
> [0] http://www.oreillynet.com/ruby/blog/2006/04/nubygems_symbolic_starvation.html
> 
> 
Well, I downloaded the PDF. It turns out they interact directly with the
OS memory manager, in this case the frightfully ancient 2.4.20 Linux
kernel. Even RHEL 3, the last Red Hat Linux to use a 2.4 kernel, has a
better memory manager than that! But yeah, it's more or less mandatory
that you talk to the OS. Also, they were benchmarking on the Jikes JVM
from IBM, not the Sun JVM and not *any* Ruby.

If I get that far with my profiling, I'll certainly take a look at
potential hacks for tuning the Ruby GC against a *real* kernel (2.6.22
and counting.) :) But there are other interesting places in Ruby to tune
besides the garbage collector, and I think KRI has a better GC anyhow.