From: Brandon Casci <brandon@...>
Date: 2007-01-04T01:31:41+09:00
Subject: Re: Memory Leak Madness

Robert Klemme wrote:
> 
> Is there some point where it stabilizes after a day or so?
> 

Nope..it keeps going like...like pacman. *wocka* *wocka* *wocka*. 
Although it does hover at various intervals before it grows, and before 
I made my changes it would grow pretty steady. When you watch the memory 
count for the process, it takes 3 steps forward, and two steps back but 
the end result is always an increase.

Good times!!

> I definitively would not add a thread that does GC.  The interpreter
> will take care of this.

Ya...I took that out. It felt wrong to do, but I wanted to see what 
would happen.


> 
> I'd probably do some object statistics (per class) and also summarize
> String, Array and Hash sizes.  

I'm pretty sure whats what the MemoryProfiler here does:
http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails

It takes that information, and writes the changes to log files at 
specified intervals.

Here is what my daemon does....
It has a couple threads, Thread1 one gets work, and Thread2 performs 
work

Thread 1:

-Thread1 gets rows from a database every N seconds
-it builds a series of objects based on the database rows, and appends 
them to an array. They are called "work items". The array is protected 
by a mutex-sync because Thread2 picks items off of it.
-it sleeps for a while then repeats the process

Thread 2:

- Thread two gets a "work item" from the above array. An item is 
"popped" off the array by calling .shift from within a mutex-sync block
- Hpricot pulls information down from url's inside the "work item"
- the information gets put put into hashes, which get dumped to disk via 
YAML::dump

And that's it. It's a pretty small daemon.











-- 
Posted via http://www.ruby-forum.com/.