[ruby-core:115580] [Ruby master Bug#20022] GC.verify_compaction_references does not actually move all objects

Issue #20022 has been updated by kjtsanaktsidis (KJ Tsanaktsidis).


Alright, I have figured it out I think - @peterzhu2118 do you mind taking another look at my PR? The insight is we need to add pages to make all size pools have the same number of pages (so their compact cursor can all advance the same number of times), and _then_ add the actual new pages we're going to move objects into (so that there's guaranteed to be space in un-compacted-yet pages for all objects to move).

----------------------------------------
Bug #20022: GC.verify_compaction_references does not actually move all objects
https://bugs.ruby-lang.org/issues/20022#change-105513

* Author: kjtsanaktsidis (KJ Tsanaktsidis)
* Status: Open
* Priority: Normal
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
While debugging https://bugs.ruby-lang.org/issues/20021, I ran into a separate issue which I figured was worth resolving whilst I had it in my head.

The intention of GC.verify_compaction_references is, I believe, to force every single movable object to be moved, so that it's possible to debug native extensions which not correctly updating their references to objects they mark as movable (if this is _not_ the case, you can stop reading now and accept my apologies!)

To do this, it doubles the number of allocated pages for each size pool, and sorts the heap pages so that the free ones are swept first; thus, every object in an old page should be moved into a free slot in one of the new pages.

This worked fine until movement of objects between size pools during compaction was implemented. That causes some problems for verify_compaction_references:

- We were doubling the number of pages in each size pool, but actually if some objects need to move into a different pool, there's no guarantee that they'll be enough room in that one.
- It's possible for the sweep & compact cursors to meet in one size pool before all the objects that want to move into that size pool from another are processed by the compaction.

You can see these problems by changing some of the movement tests in test_gc_compact.rb to try and move e.g. 50,000 objects instead of 500 (by changing `HASH_COUNT`); the test is not able to actually move all of the objects in a single compaction run.

What I implemented in this PR (https://github.com/ruby/ruby/pull/9041) is two things:

- Firstly, it examines every object and determine where it wants to be compacted into; we use this information to calculate the desired number of pages to add to each size pool.
- Secondly, it compacts twice; once in order of ascending size pool, and once descending. This means that we are guaranteed to be able to move everything we want to move into a size pool before we start advancing that pool's own compact cursor.

With these fixes in place, I was able to make the compaction tests move any amount of objects (previously, `test_moving_hashes_down_size_pools` would begin failing on my machine if I set `HASH_COUNT` to above about 6,000.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/