From: Cliff Cyphers Date: 2006-08-26T03:36:03+09:00 Subject: Re: Can Anyone Explain This Memory Leak? This isn't considered bizarre behavior and in fact is a feature. Even after you shutdown an application it keeps shared libs and such in memory (cache). That way the system as a whole is much quicker when you access applications that call these shared libs. No point in having the overhead of re-loading things into memory over and over again. If you get to the point where you use additional applications and there isn't free ram it flushes some of the cached objects out so that you don't have to swap. M. Edward (Ed) Borasky wrote: > Zed Shaw wrote: >> On Fri, 2006-08-25 at 14:33 +0900, ara.t.howard@noaa.gov wrote: >>> On Fri, 25 Aug 2006, Zed Shaw wrote: >>> >>>> Hi Folks, >>>> >>>> Sorry to get your attention. :-) >>>> >>>> There's a very strange problem with Mongrel where if Threads are created >>>> because of the Mutex around Rails dispatching, then lots of ram gets >>>> created that never seems to go away. >>>> >>>> I boiled the problem down to this: >>>> >>>> http://pastie.caboo.se/10194 >>>> >>> hi zed- >>> >>> i don't think you have a leak. try running under electric fence (ef). when i >>> do i clearly see the memory rise from 1->20% on my desktop, and then decline >>> back down to 1%, over and over with no reported leaks. the cycle matches >>> the logging of the script perfectly. >>> >>> here's the thing though, when i don't run it under electric fence i see the >>> memory climb to about 20% and then stay there forever. but this too does not >>> indicate a leak. it just shows how calling 'free' in a process doesn't really >>> release memory to the os, only to the process itself. the reason you see the >>> memory vary nicely under ef is that it replaces the standard malloc/free with >>> it's own voodoo - details of which i do not understand or care too. the >>> point, however, is that it's 'free' which is doing the 'leaking' - just at the >>> os level, not the process (ruby) level. we have tons of really long running >>> processes that exhibit the exact same behaviour - basically the memory image >>> will climb to maximum and stay there. oddly, however, when you tally them all >>> up the usage exceeds the system capacity plus swap by miles. >>> >> Nope, I can't agree with this because the ram goes up, the OS will kill >> it eventually, and if I remove the guard the ram doesn't do this. >> >> And where are you getting your information that free doesn't free >> memory? I'd like to read that since all my years of C coding says that >> is dead wrong. Care to tell me how malloc/free would report 80M with >> Mutex but properly show the ram go down when there is no-Mutex? >> >> And why Linux would kill processes if the ram get too high? Why whole >> VPS servers crash? I mean if this ram was just "fake" reporting (which >> is very hard to believe) then why are all these things happening? >> >> So please, point me at where in the specifications for malloc/free on >> Linux it says that the memory reported will be high even though free and >> malloc is called on 80M of ram slowly cycled out, and that linux will >> still kill your process even though this ram is not really owned by the >> process. >> >> > > I've had a lot of experience with the "bizarre" behavior of the Linux > memory manager. First of all, which kernel do you have? Second, the > Linux out-of-memory killer can be turned off (also kernel-dependent). > Finally, Linux has this philosophy that "free memory is wasted memory", > and "I am Linux, you are the user, I know what's good for me, if that > works for you, great!" The combination of these means that things that > make sense to you or to a performance engineer might not actually be > happening. :) > > Oh, yeah, how much physical RAM do you have? If you have a recent > kernel, which of the dozens of memory configuration options are you > using? Have you considered switching to another OS? :) >