[ruby-core:81767] [Ruby trunk Bug#11101] Forking is killing my memory when running GC

Issue #11101 has been updated by olivierlacan (Olivier Lacan).


normalperson (Eric Wong) wrote:
>  Did you try the nakayoshi_fork gem as ko1 suggested?

Sorry for replying so late. I think I missed the notification when you originally replied.
I believe nakayoshi_fork did help a bit to bring response times near to their pre 2.2 
averages.

>  Performance is unlikely to be twice as slow unless you're using enough
>  memory to hit swap...

To be fair, it wasn't. I think we had a 30 ms average that jumped to 60 at first and 
leveled of at 50 ms with nakayoshi_fork. I don't believe we were close to hitting swap, and 
I think I was seeing this on multiple different servers at the time.

How big are your processes on 2.1 and 2.2?

I'm not sure what you're asking there.

>  Is this on 64-bit?
Yes.

>  What else can you tell us about your setup?
>  (malloc used, GC tunings, etc).

We're not using any GC tunings and running with Passenger Enterprise on Debian 8. No jemalloc if that's what you're wondering.

`passenger_max_pool_size: 16`
`max_instances: 12`

Interestingly, upgrading to 2.3.4 brought us back to pre-2.2 response time averages of about 35 ms, and the averages are much more stable as well.


----------------------------------------
Bug #11101: Forking is killing my memory when running GC
https://bugs.ruby-lang.org/issues/11101#change-65464

* Author: tkalmus (Thomas Kalmus)
* Status: Feedback
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version: 
* ruby -v: 2.2.1
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------

Memory is duplicated when forking and calling GC in 2.2.1. The main issue here is that we are operating with huge data, going up to 3GB, and one fork is enough to kill our machine.

We have written a small program that reproduces the issue (see attached file).

    The program instantiates an object and then forks into two processes.
    The GC is called in the child process.
    The memory allocation (as given by /proc/pid/smaps) changes from shared to private, thereby indicating a doubling of memory consumption.

Here is the output of the program (size is in mb):

ruby version 2.1.3
   time   pid message             shared    private
 4.011s  4723 Parent pre GC           68          0
 4.013s  4737 Child  pre GC           68          0
 8.019s  4723 Parent post GC           5         62
 8.093s  4737 Child  post GC           5         66

We have tested the program on Ubuntu 14.04 with ruby 1.9.3 and 2.2.1. The tests have been performed on a freshly installed Ubuntu machine.

We have also tried to fork 10 children and see a 10 doubling of the memory consumption, the issue only occurs after running the GC.


---Files--------------------------------
mem.rb (1.25 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>