[ruby-core:62735] [ruby-trunk - Feature #9113] Ship Ruby for Linux with jemalloc out-of-the-box

Issue #9113 has been updated by Eric Wong.


 KOSAKI Motohiro <[e-mail]> wrote:
 > Do we have only one benchmark provided by Sam? I don't think it is
 > enough much comparison to make decision.
 
 Empirical evidence on my 32-bit yahns server after running several days
 shows ~40M RSS w/ jemalloc 3.0/3.6 vs 60-80M RSS for eglibc malloc on
 Debian stable.  I'll work on gathering more data for other systems.
 
 > Anyway, I don't think Linux distros enable jemalloc because of their
 > packaging policy even if we enable by default.
 
 I know distros do not like bundling extra libs, but I'm not aware of
 policies against extra linkage for already packaged libraries.
 
 Looking at Debian stable packages, both redis and varnish packages[1]
 use jemalloc on common architectures.  For redis, Debian just favors the
 system jemalloc instead of the bundled one.
 
 > So we need, at least
 > 1. A way to disable it
 
 My patch respects the --without-jemalloc option
 
 > 2. Easy to detect which malloc is used from bug reports. For our maintenance.
 
 rb_bug already shows dynamically loaded libraries.  My patch only enables dynamic
 link by default.
 
 [1] http://http.us.debian.org/debian/pool/main/r/redis/redis_2.4.14-1.dsc
 http://http.us.debian.org/debian/pool/main/r/redis/redis_2.4.14.orig.tar.gz
 http://http.us.debian.org/debian/pool/main/r/redis/redis_2.4.14-1.debian.tar.gz
 http://http.us.debian.org/debian/pool/main/v/varnish/varnish_3.0.2-2+deb7u1.dsc
 http://http.us.debian.org/debian/pool/main/v/varnish/varnish_3.0.2.orig.tar.gz
 http://http.us.debian.org/debian/pool/main/v/varnish/varnish_3.0.2-2+deb7u1.debian.tar.gz

----------------------------------------
Feature #9113: Ship Ruby for Linux with jemalloc out-of-the-box
https://bugs.ruby-lang.org/issues/9113#change-46859

* Author: Sam Saffron
* Status: Feedback
* Priority: Normal
* Assignee: 
* Category: build
* Target version: 
----------------------------------------
libc's malloc is a problem, it fragments badly meaning forks share less memory and is slow compared to tcmalloc or jemalloc. 

both jemalloc and tcmalloc are heavily battle tested and stable. 

2 years ago redis picked up the jemalloc dependency see: http://oldblog.antirez.com/post/everything-about-redis-24.html 

To quote antirez:
``
But an allocator is a serious thing. Since we introduced the specially encoded data types Redis started suffering from fragmentation. We tried different things to fix the problem, but basically the Linux default allocator in glibc sucks really, really hard. 
``

--- 

I recently benched Discourse with tcmalloc / jemalloc and default and noticed 2 very important thing: 

median request time reduce by up to 10% (under both)
PSS (proportional share size) is reduced by 10% under jemalloc and 8% under tcmalloc.

We can always use LD_PRELOAD to yank these in, but my concern is that standard distributions are using a far from optimal memory allocator. It would be awesome if the build, out-of-the-box, just checked if it was on Linux  (eg: https://github.com/antirez/redis/blob/unstable/src/Makefile#L30-L34 ) and then used jemalloc instead. 

---Files--------------------------------
0001-configure.in-add-with-jemalloc-option.patch (1.29 KB)


-- 
https://bugs.ruby-lang.org/