From: "M. Edward (Ed) Borasky" <znmeb@...>
Date: 2006-08-26T12:24:55+09:00
Subject: Off topic Linux memory management rant (was Re: Can Anyone Explain This Memory Leak?)

Cliff Cyphers wrote:
> This isn't considered bizarre behavior and in fact is a feature.  Even
> after you shutdown an application it keeps shared libs and such in
> memory (cache).  That way the system as a whole is much quicker when you
> access applications that call these shared libs.  No point in having the
> overhead of re-loading things into memory over and over again.  If you
> get to the point where you use additional applications and there isn't
> free ram it flushes some of the cached objects out so that you don't
> have to swap.
> 
>

As a performance engineer who works with managers and capacity planners
of large systems, I consider a memory manager that can't be tuned to a
workload bizarre. Performance comes in many dimensions, and we ask our
Linux systems to serve numerous roles.

Sometimes we want them to process large batch jobs for rapid turnaround,
sometimes we want them to provide rapid interactive response to
thousands of people logged on to an application, sometimes we want them
to manage a huge DBMS, sometimes we want them to serve up web pages,
sometimes we want them to participate in a high-performance cluster, and
sometimes we want them to be scientific workstations. The idea that a
single memory management algorithm/code without tuning parameters can
serve all of those needs is ludicrous.

I don't know if the 2.6 kernel is genuinely better than 2.4, I've gotten
smarter about the Linux memory manager, or both. :) What I *do* know is
that just about all forms of the 2.4 Linux kernel, even the carefully
tuned ones in RHEL 3, behave in unpredictable and less than useful ways
when people do even moderately stupid things. I think a server ought to
be able to deal with a memory leak in a web server application in a more
productive way than the out of memory killer!