[ruby-core:84593] [Ruby trunk Bug#14275] GC not aggressive enough

Issue #14275 has been reported by normalperson (Eric Wong).

----------------------------------------
Bug #14275: GC not aggressive enough
https://bugs.ruby-lang.org/issues/14275

* Author: normalperson (Eric Wong)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Ruby memory usage has always been a thorn in my side, more so than (lack of)
speed.  I sometimes take vacations into Perl5 or C to escape from it.

Sometimes a well-placed String#clear has a good effect, but I'm not sure
if it's acceptable for Rubyists to sprinkle throughout their code.

I've made some changes to net/http for 2.5 to improve the situation:

https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=58840
https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=58846

And proposed one more for 2.6: https://bugs.ruby-lang.org/issues/14268

But a String#clear is still necessary for the end user.  Even
with my patch for [Feature #14268], that example takes around
10x more memory without String#clear.

Can Ruby be better out-of-the box?

The attached script takes about 120MB on my system (without String#clear)

Lazy sweep seems to hurt in this case, because large (16K)
buffers are floating around unused with delayed free(3).
Sweeping immediately for malloc increase (and not other GC
trigger points) reduces memory from around 120 MB to 75 MB.

```
--- a/gc.c
+++ b/gc.c
@@ -7825,7 +7825,7 @@ objspace_malloc_increase(rb_objspace_t *objspace, void *mem, size_t new_size, si
 		gc_rest(objspace); /* gc_rest can reduce malloc_increase */
 		goto retry;
 	    }
-	    garbage_collect_with_gvl(objspace, FALSE, FALSE, FALSE, GPR_FLAG_MALLOC);
+	    garbage_collect_with_gvl(objspace, FALSE, FALSE, TRUE, GPR_FLAG_MALLOC);
 	}
     }
 
```

Maybe more could be done and less intrusively.  Providing a
custom custom malloc (not just wrapper API) would allow us to
finish sweeping before calls to mmap(2)/sbrk(2) are necessary to
request memory from the kernel.  That might be a maintenance
nightmare...


---Files--------------------------------
net-http-readbody.rb (642 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>