From: dennisb55@... Date: 2018-05-18T03:13:28+00:00 Subject: [ruby-core:87155] [Ruby trunk Feature#14718] Use jemalloc by default? Issue #14718 has been updated by bluz71 (Dennis B). I have taken Mike's script, increased the `THREAD_COUNT` to 20 and run it on my Linux box (Intel i5-4590 quad-core, 16GB RAM, Linux Mint 18.3 with kernel 4.15.0). glibc malloc results: ~~~ % time MALLOC_ARENA_MAX=1 ruby frag.rb VmRSS: 4,164,108 kB real 12.001s % time MALLOC_ARENA_MAX=2 ruby frag.rb VmRSS: 4,362,360 kB real 12.259s % time MALLOC_ARENA_MAX=3 ruby frag.rb VmRSS: 4,486,620 kB real 12.618s % time MALLOC_ARENA_MAX=4 ruby frag.rb VmRSS: 4,692,404 kB real 12.064s % time MALLOC_ARENA_MAX=5 ruby frag.rb VmRSS: 4,682,908 kB real 12.364s % time MALLOC_ARENA_MAX=6 ruby frag.rb VmRSS: 4,914,360 kB real 12.249s % time MALLOC_ARENA_MAX=7 ruby frag.rb VmRSS: 5,063,128 kB real 12.507s % time MALLOC_ARENA_MAX=8 ruby frag.rb VmRSS: 5,325,748 kB real 12.598s % time MALLOC_ARENA_MAX=12 ruby frag.rb VmRSS: 5,965,244 kB real 12.321s % time MALLOC_ARENA_MAX=16 ruby frag.rb VmRSS: 6,565,000 kB real 11.827s % time MALLOC_ARENA_MAX=24 ruby frag.rb VmRSS: 7,106,460 kB real 12.007s % time MALLOC_ARENA_MAX=32 ruby frag.rb VmRSS: 7,101,580 kB real 11.646s % time ruby frag.rb VmRSS: 7,241,108 kB real 11.739s ~~~ And with jemalloc 3.6.0 (1st stanza) and 5.0.1 (2nd stanza): ~~~ % time LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so ruby frag.rb VmRSS: 4,680,260 kB real 25.103s % time LD_PRELOAD=/home/bluz71/.linuxbrew/Cellar/jemalloc/5.0.1/lib/libjemalloc.so ruby frag.rb VmRSS: 7,162,508 kB real 12.111s ~~~ I am not sure if this test is really showing memory fragmentation or arena count expense. Mike's test clearly runs twice as slow via jemalloc 3.6.0. Conversely when run with jemalloc 5.0.1 the performance issue goes away but the `VmRSS` size matches high arena count glibc malloc. I do use jemalloc 3.6.0 with my Rails application with great results. Noah Gibbs noted excellent results when using jemalloc with his Rails benchmark tests as noted [here](http://engineering.appfolio.com/appfolio-engineering/2018/2/1/benchmarking-rubys-heap-malloc-tcmalloc-jemalloc). Nevertheless we should indeed be cautious. I tend to agree now with backing off using jemalloc by default since different versions have wildly different results with Mike's test. ---------------------------------------- Feature #14718: Use jemalloc by default? https://bugs.ruby-lang.org/issues/14718#change-72145 * Author: mperham (Mike Perham) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7. I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively. Some data points: https://twitter.com/brandonhilkert/status/987400365627801601 https://twitter.com/d_jones/status/989866391787335680 https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469 Redis moved to jemalloc many years ago and it solved all of their memory issues too. Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem. -- https://bugs.ruby-lang.org/ Unsubscribe: