From: Christopher Dicely <cmdicely@...>
Date: 2009-05-08T04:02:05+09:00
Subject: Re: Ruby memory usage

On Thu, May 7, 2009 at 11:01 AM, Pete Hodgson <ruby-forum@thepete.net> wrote:
> Hi Folks,
>
> I'm trying to figure out a memory usage issue with some long running
> ruby processes and was wondering if anyone can help me out.
>
> The background is that I have a set of long running ruby processes which
> are communicating with a larger system using an activeMQ server via
> STOMP. These processes are running for weeks at a time, most of the time
> just sitting there waiting for an incoming message.
>
> My team has noticed that these processes memory usage gradually grows
> over time. As a concrete example one of my processes starts life using
> around ~20MB of virtual memory, and 20 hours later is sitting at 382MB
> of virtual memory, 115MB resident.
>
> My first step in trying to figure this out was to use a slightly tweaked
> version of the memory profiler described here:
> http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails.
> Essentially I'm just taking a snapshot of ObjectSpace every 10 seconds
> and dumping a top 20 count of how many instances there are of a class.
> Here are two snapshots, one take a few minutes after the process starts,
> and one ~20 hours later:
>
> ----------Wed May 06 14:03:33 -0700 2009----------
> Top 20
>   23724: String
>    1121: Hash
>     776: Class
>     738: Regexp
>     578: Array
>     474: StatsBucket(id: integer, organization_id: integer,
> se_account_id: integer, date: date)
>     474: TZInfo::TimezoneTransitionInfo
>     374: Proc
>     236: Module
>     154: Bignum
>     152: UnboundMethod
>     134: Range
>      62: Rational
>      57: Float
>      34: Gem::Version
>      28: ActiveRecord::ConnectionAdapters::MysqlColumn
>      27: Time
>      22: Gem::Version::Requirement
>      17: Mutex
>      16: Gem::Specification
>
>
> ----------Thu May 07 10:45:48 -0700 2009----------
> Top 20
>   23848: String
>     834: Class
>     790: Bignum
>     783: Array
>     738: Regexp
>     506: Hash
>     474: TZInfo::TimezoneTransitionInfo
>     376: Proc
>     236: Module
>     159: StatsBucket(id: integer, organization_id: integer,
> se_account_id: integer, date: date)
>     152: UnboundMethod
>     134: Range
>     129: Rational
>     107: Float
>     104: MatchData
>      54: Thread
>      46: Timeout::Error
>      45: Time
>      34: Gem::Version
>      32: ActiveRecord::ConnectionAdapters::MysqlColumn
>
>
>
> As you can see there's not that much of a change in the ObjectSpace, and
> yet the memory usage of this same process has blown up from 20MB to over
> 300MB of virtual memory.
>
> Can anyone give me any clues on what's going on here? Or any pointers as
> to what else I can in terms of measurements and instrumentation to get a
> better insight as to what's going here?

The first thing that jumps to mind is that the 54 instance of Thread
in the later listing may be a sign of the problem; unless you really
should have that many Threads hanging around, you may be doing
something that is preventing used Thread objects from being garbage
collected. If you are, for instance, launching a new thread when you
get a message and they are hanging around after they aren't needed,
that could explain the expanding memory usage.