From: Pete Hodgson <ruby-forum@...>
Date: 2009-05-08T03:01:20+09:00
Subject: Ruby memory usage

Hi Folks,

I'm trying to figure out a memory usage issue with some long running
ruby processes and was wondering if anyone can help me out.

The background is that I have a set of long running ruby processes which
are communicating with a larger system using an activeMQ server via
STOMP. These processes are running for weeks at a time, most of the time
just sitting there waiting for an incoming message.

My team has noticed that these processes memory usage gradually grows
over time. As a concrete example one of my processes starts life using
around ~20MB of virtual memory, and 20 hours later is sitting at 382MB
of virtual memory, 115MB resident.

My first step in trying to figure this out was to use a slightly tweaked
version of the memory profiler described here:
http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails.
Essentially I'm just taking a snapshot of ObjectSpace every 10 seconds
and dumping a top 20 count of how many instances there are of a class.
Here are two snapshots, one take a few minutes after the process starts,
and one ~20 hours later:

----------Wed May 06 14:03:33 -0700 2009----------
Top 20
   23724: String
    1121: Hash
     776: Class
     738: Regexp
     578: Array
     474: StatsBucket(id: integer, organization_id: integer,
se_account_id: integer, date: date)
     474: TZInfo::TimezoneTransitionInfo
     374: Proc
     236: Module
     154: Bignum
     152: UnboundMethod
     134: Range
      62: Rational
      57: Float
      34: Gem::Version
      28: ActiveRecord::ConnectionAdapters::MysqlColumn
      27: Time
      22: Gem::Version::Requirement
      17: Mutex
      16: Gem::Specification


----------Thu May 07 10:45:48 -0700 2009----------
Top 20
   23848: String
     834: Class
     790: Bignum
     783: Array
     738: Regexp
     506: Hash
     474: TZInfo::TimezoneTransitionInfo
     376: Proc
     236: Module
     159: StatsBucket(id: integer, organization_id: integer,
se_account_id: integer, date: date)
     152: UnboundMethod
     134: Range
     129: Rational
     107: Float
     104: MatchData
      54: Thread
      46: Timeout::Error
      45: Time
      34: Gem::Version
      32: ActiveRecord::ConnectionAdapters::MysqlColumn



As you can see there's not that much of a change in the ObjectSpace, and
yet the memory usage of this same process has blown up from 20MB to over
300MB of virtual memory.

Can anyone give me any clues on what's going on here? Or any pointers as
to what else I can in terms of measurements and instrumentation to get a
better insight as to what's going here?

Any help greatly appreciated!

Cheers,
Pete
-- 
Posted via http://www.ruby-forum.com/.