From: Eric Wong <normalperson@...>
Date: 2011-09-15T05:02:02+09:00
Subject: Re: Find memory leak in very complex Ruby app

Tsyren Ochirov <tsyren.hey@gmail.com> wrote:
> So, my question is how to find memory leak in very complex Ruby app and
> not put all my life into this work?

In the absence-of/incompatibility of specialized tools or alternative
runtimes, an old-fashioned divide-and-conquer method works (regardless
of programming language/implementation):

1) loop your app in some way that provably leaks memory

2) Cut the code paths your app follows in half and have your loop
   follow one of them.

   a) First, split out the types of inputs your app receives from
      users (e.g. for an HTTP app, maybe all GET requests to a
      certain path).

   b) Neutralize code paths by editing them out.  You can do this by
      placing early returns (and mocking/memoizing expected return
      values, if any) or commenting out code (including wrapping it
      with "if false; ...; end"

      Your app doesn't have to be correct while you're doing this,
      it just has to be able to loop successfully.

3) Repeat 1) with half your code active.  If the new loop still leaks
   memory, repeat 2) to divide the code up into smaller chunks, and keep
   repeating until you find it.  If it doesn't leak memory, follow the
   code path you didn't take.

A few notes:

* You can put the loop described in (1) in the app itself to speed
  things up (e.g. to avoid slow I/O).

* Reproducing the leak should get faster as the possible code paths to
  a leak become smaller.

* You may find multiple sources of leaks this way.

* You may need to go all the way down to C code (in Ruby itself or an
  extension) to find the bug.

* If you know your way around the app and already know /exactly/
  which code paths do not leak memory, you can neutralize those code
  paths to speed things up.