From: normalperson@... Date: 2018-01-03T01:52:59+00:00 Subject: [ruby-core:84593] [Ruby trunk Bug#14275] GC not aggressive enough Issue #14275 has been reported by normalperson (Eric Wong). ---------------------------------------- Bug #14275: GC not aggressive enough https://bugs.ruby-lang.org/issues/14275 * Author: normalperson (Eric Wong) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: * Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN ---------------------------------------- Ruby memory usage has always been a thorn in my side, more so than (lack of) speed. I sometimes take vacations into Perl5 or C to escape from it. Sometimes a well-placed String#clear has a good effect, but I'm not sure if it's acceptable for Rubyists to sprinkle throughout their code. I've made some changes to net/http for 2.5 to improve the situation: https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=58840 https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=58846 And proposed one more for 2.6: https://bugs.ruby-lang.org/issues/14268 But a String#clear is still necessary for the end user. Even with my patch for [Feature #14268], that example takes around 10x more memory without String#clear. Can Ruby be better out-of-the box? The attached script takes about 120MB on my system (without String#clear) Lazy sweep seems to hurt in this case, because large (16K) buffers are floating around unused with delayed free(3). Sweeping immediately for malloc increase (and not other GC trigger points) reduces memory from around 120 MB to 75 MB. ``` --- a/gc.c +++ b/gc.c @@ -7825,7 +7825,7 @@ objspace_malloc_increase(rb_objspace_t *objspace, void *mem, size_t new_size, si gc_rest(objspace); /* gc_rest can reduce malloc_increase */ goto retry; } - garbage_collect_with_gvl(objspace, FALSE, FALSE, FALSE, GPR_FLAG_MALLOC); + garbage_collect_with_gvl(objspace, FALSE, FALSE, TRUE, GPR_FLAG_MALLOC); } } ``` Maybe more could be done and less intrusively. Providing a custom custom malloc (not just wrapper API) would allow us to finish sweeping before calls to mmap(2)/sbrk(2) are necessary to request memory from the kernel. That might be a maintenance nightmare... ---Files-------------------------------- net-http-readbody.rb (642 Bytes) -- https://bugs.ruby-lang.org/ Unsubscribe: