From: shyouhei@... Date: 2017-01-20T03:33:55+00:00 Subject: [ruby-core:79174] [Ruby trunk Feature#12967] Add a default for RUBY_GC_HEAP_GROWTH_MAX_SLOTS out-of-the-box Issue #12967 has been updated by Shyouhei Urabe. We looked at this issue at yesterday's developer meeting. Ko1 said he was not sure if the proposed default value is decent. Also he said to me that there should be some kind of comprehensive approach to parameterize GC. It seems he thinks sporadic use of environment variables make things complicated. ---------------------------------------- Feature #12967: Add a default for RUBY_GC_HEAP_GROWTH_MAX_SLOTS out-of-the-box https://bugs.ruby-lang.org/issues/12967#change-62598 * Author: Sam Saffron * Status: Open * Priority: Normal * Assignee: Koichi Sasada * Target version: ---------------------------------------- As it stands Ruby heaps grow at a rate that is too fast for the vast majority of applications that use Ruby. ``` def rss _,rss = `ps ax -o pid,rss | grep '#{Process.pid} '`.split(/\s+/) rss.to_i end def heap_length GC.stat[:heap_sorted_length] end @array = [] @heap_length = heap_length while true @array << "" if @heap_length < heap_length @heap_length = heap_length puts "RSS #{rss}, Heap length: #{@heap_length}" end end ``` ``` RSS 14364, Heap length: 133 RSS 33828, Heap length: 237 RSS 39652, Heap length: 424 RSS 59476, Heap length: 759 RSS 112192, Heap length: 1342 RSS 126452, Heap length: 2257 RSS 142020, Heap length: 3295 RSS 184056, Heap length: 5058 RSS 266648, Heap length: 7810 RSS 339224, Heap length: 11113 ``` compare that to a sane default of say RUBY_GC_HEAP_GROWTH_MAX_SLOTS=100000 ``` RUBY_GC_HEAP_GROWTH_MAX_SLOTS=100000 ruby test_mem.rb RSS 15580, Heap length: 133 RSS 30452, Heap length: 237 RSS 37212, Heap length: 424 RSS 55116, Heap length: 667 RSS 107716, Heap length: 911 RSS 114388, Heap length: 1095 RSS 117492, Heap length: 1233 RSS 119196, Heap length: 1309 RSS 122872, Heap length: 1474 RSS 123588, Heap length: 1509 RSS 125396, Heap length: 1589 RSS 129648, Heap length: 1756 RSS 131096, Heap length: 1781 RSS 132812, Heap length: 1863 RSS 136420, Heap length: 2031 RSS 137960, Heap length: 2052 RSS 139832, Heap length: 2135 RSS 143568, Heap length: 2304 RSS 145112, Heap length: 2322 RSS 146936, Heap length: 2406 RSS 150404, Heap length: 2573 RSS 173660, Heap length: 2781 RSS 175984, Heap length: 2866 RSS 179664, Heap length: 3034 RSS 192980, Heap length: 3242 RSS 193784, Heap length: 3325 RSS 197304, Heap length: 3493 ``` Note, this test deals with the absolute minimal amount of RSS, cause all data is stored in RVALUES, with strings like typical apps RSS can grow in much bigger increments. RSS will jump from 266MB to 339MB, out of the box, this is a massive amount of growth just for Ruby heaps. The trouble with this huge amount of heap growth is that it amplifies issues around heap fragmentation, in web apps once heap grows it is very unlikely it will shrink again cause there are just enough long living objects that a full slot never frees up properly. My suggestion is to ship with a far safer default of: RUBY_GC_HEAP_GROWTH_MAX_SLOTS=100000 This has zero impact on "small scripts" < 50M rss, and major advantages for larger apps. Thoughts? Honestly I see zero downsides to capping it so Ruby heaps only grow in 5mb increments. It eliminates lots of current bloat and allows the GC to operate more efficiently. -- https://bugs.ruby-lang.org/ Unsubscribe: