From: Toby DiPasquale <toby@...>
Date: 2006-03-18T01:29:59+09:00
Subject: Re: Writing long-running daemons without memory leaks?

unknown wrote:
> i'm running dozens of ruby daemons, some of which are extremely (months 
> at a
> time) long lived and have not seen any gc issues.  can you post a 
> specific
> (minimal) example that you find leaks memory?

Sure. Here is a server and client, resp, that exhibit the behavior I am 
referring to:

puma:> cat leak.rb
# vim:set ts=4 sw=4 ai:
require 'socket'

class Buffer
    def initialize size
        @size = size
        @buffer = []
        @length = 0
    end
    attr_reader :length, :size
    def full?  ; @length == @size end
    def empty? ; @length.zero? end
    # Lets you fill up to capacity and then returns
    # what's left over
    def fill data
        if full?
            data
        elsif @length + data.length < @size
            @buffer << data
            @length += data.length
            nil
        else
            l = @size - @length
            @buffer << data[0, l]
            data[l..-1]
        end
    end
    def expunge
        buf = @buffer.join
        @buffer.clear
        @length = 0
        buf
    end
end     # class Buffer

def read_long s
    s.read( 4).unpack( "N")[0]
end

# main client handling thread logic
client_handler = lambda do |s|
    begin
        len = read_long s
        buf = Buffer.new len
        until buf.full?
            x = s.read 4096
            break if x.nil?
            buf.fill x
        end
        puts "writing #{buf.length} bytes"
        f = File.open "/dev/null", "w"
        f.write buf.expunge
        f.close
    ensure
        s.close
    end
end

# main server loop
ss = TCPServer.new '127.0.0.1', 10001
begin
    while true
        s = ss.accept
        puts "got connection"
        Thread.start s, &client_handler
    end
ensure
    ss.close
end

puma:~> cat leak_client.rb
# vim:set ts=4 sw=4 ai:
require 'socket'

def write_long s, l
        s.write( [l].pack( "N"))
end

str = "a" * 65536

t = TCPSocket.new '127.0.0.1', 10001
write_long t, 1024 * str.length
1024.times { t.write str }
t.close

puma:~>

And here's the output from a simple test to show the issue:

puma:~> for i in 1 2 3 4 5 6 7 8 9 10; do ps aux | grep [l]eak.rb ; ruby 
leak_client.rb ; done
toby      8452  0.0  0.1   3116  1684 pts/3    S+   11:23   0:00 ruby 
leak.rb
toby      8452  3.8 12.8 143032 133432 pts/3   Sl+  11:23   0:01 ruby 
leak.rb
toby      8452  6.4 15.8 173664 164020 pts/3   Sl+  11:23   0:02 ruby 
leak.rb
toby      8452  7.8 15.2 167988 158472 pts/3   Sl+  11:23   0:03 ruby 
leak.rb
toby      8452  9.2 15.8 173532 164004 pts/3   Sl+  11:23   0:03 ruby 
leak.rb
toby      8452 10.7 15.2 168024 158508 pts/3   Sl+  11:23   0:04 ruby 
leak.rb
toby      8452 12.0 15.8 173568 164000 pts/3   Sl+  11:23   0:04 ruby 
leak.rb
toby      8452 13.2 15.8 173568 164008 pts/3   Sl+  11:23   0:05 ruby 
leak.rb
toby      8452 14.3 15.8 173568 164012 pts/3   Sl+  11:23   0:06 ruby 
leak.rb
toby      8452 15.7 15.0 165012 155496 pts/3   Sl+  11:23   0:06 ruby 
leak.rb
puma:~> ps aux | grep [l]eak.rb
toby      8452 13.0 22.1 239132 229544 pts/3   Sl+  11:23   0:07 ruby 
leak.rb
puma:~>

This memory never really goes away. Also notice that the client is 
sending 64MB, but the first time the leak.rb image jumps to 143MB. Any 
clues? Am I just doing something really stupid? TIA.

--
Toby DiPasquale

-- 
Posted via http://www.ruby-forum.com/.