[ruby-core:118691] [Ruby master Misc#20652] Memory allocation for gsub has increased from Ruby 2.7 to 3.3
From:
"shyouhei (Shyouhei Urabe) via ruby-core" <ruby-core@...>
Date:
2024-07-26 07:19:47 UTC
List:
ruby-core #118691
Issue #20652 has been updated by shyouhei (Shyouhei Urabe).
I agree this is counter-intuitive. The #17507 problem was that
```ruby
i = lambda { ...(touches $~)... }
many.times { Thread.start { many.times { i.call } } }
```
would break. This is because `$~` is `i`'s local variable. `i` is shared across threads so is `$~`.
@jeremyevans0 fixed this so that no two `$~`s are identical, no matter multi threaded situation or not.
The fix of course increases memory usage. But because the root cause of the problem is sharing local variables across threads, to unshare them the bloat seems kind of inevitable to me. I also think this is a needed fix. Nobody thinks `gsub` modifies `$~` behind the scene.
I would agree with @byroot that it could be better if we had a `gsub` variant that doesn't touch `$~`.
----------------------------------------
Misc #20652: Memory allocation for gsub has increased from Ruby 2.7 to 3.3
https://bugs.ruby-lang.org/issues/20652#change-109227
* Author: orisano (Nao Yonashiro)
* Status: Open
* Assignee: jeremyevans0 (Jeremy Evans)
----------------------------------------
I recently upgraded from ruby 2.7.7 to 3.3.1 and noticed that the GC load increased.
When I used the allocation profiler to investigate, I found that memory allocation from gsub had increased.
The problem was code like this:
```ruby
s = "foo "
s.gsub(/ (\s+)/) { " #{' ' * Regexp.last_match(1).length}" }
```
When I compared the results of heap-profiler between 2.7.7 and 3.3.1, I found that MatchData was increasing.
https://gist.github.com/orisano/98792dee260106e9b6fcb45bbabeb1e6
https://github.com/ruby/ruby/commit/abc0304cb28cb9dcc3476993bc487884c139fd11
I discovered that the cause is this commit, which stopped reusing backref to avoid race conditions.
Is there a way to reuse backref while still avoiding race conditions?
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/