From: "wks (Kunshan Wang) via ruby-core" Date: 2026-05-13T11:40:28+00:00 Subject: [ruby-core:125490] [Ruby Feature#22067] New TypedData bit to allow the type to be freed in parallel Issue #22067 has been updated by wks (Kunshan Wang). File Screenshot_20260513_185452.webp added File Screenshot_20260513_192449.webp added File Screenshot_20260513_192734.webp added File Screenshot_20260513_192910.webp added If I read correctly, this proposal is about improving the performance of sweeping by running `obj_free` of TypedData concurrently with mutators. But I think this does not change a fact that the TypedData is still subject to finalization (in the form of `obj_free`) even with the proposed `RUBY_TYPED_PARALLEL_FREE_SAFE`. Note that CRuby has two forms of finalization (here "finalization" refers to the actions need to be done when an object is deemed unreachable): 1. `obj_free` which, in the case of `TypedData`, can be customized by providing a `dfree` pointer. 2. `ObjectSpace::define_finalizer`. If an object needs finalization in `obj_free`, its memory cannot be reclaimed until `obj_free` finishes because `obj_free` needs to read its fields. In CRuby, this leads to calling `obj_free` in lazy sweeping (in allocation slow paths), or "zombie" objects which are kept alive after a GC. If an object needs to release some resources (e.g. freeing memory, closing files, etc.) in `obj_free`, it must be done during sweeping, regardless whether it is done in the mutator thread or done concurrently in a "worker" thread. # Can we use `ObjectSpace::define_finalizer` instead? Before talking about the `RUBY_TYPED_PARALLEL_FREE_SAFE` proposal, I'd like to point out that `ObjectSpace::define_finalizer` is a better alternative. If the purpose of this proposal is optimizing for the cleaning up of TypedData types in the CRuby runtime itself, we should see if `ObjectSpace::define_finalizer` (or a similar finalization mechanism based on weak tables) is feasible. We may encourage extension writers to use `ObjectSpace::define_finalizer` if possible. The advantage of `ObjectSpace::define_finalizer` is that, because the finalizer registered with `ObjectSpace::define_finalizer` does not have access to the dead object itself, the dead object can be reclaimed regardless whether the finalizer has been executed or not. This means if a TypedData uses `ObjectSpace::define_finalizer` (instead of `dfree`) as its clean-up mechanism, it can just use `RUBY_TYPED_DEFAULT_FREE`, and `obj_free` doesn't need to invoke `dfree`, either. If the TypedData is also `RUBY_TYPED_EMBEDDABLE`, the cell can be trivially reclaimed without invoking `obj_free()` at all. Compared to concurrent sweeping which still needs to call `obj_free`, it's better to skip `obj_free` altogether. In MMTk, we skip calling `obj_free` on TypedData instances that are both `RUBY_TYPED_EMBEDDABLE` and `RUBY_TYPED_DEFAULT_FREE`. In the default GC, `obj_free` basically does no-op to such objects.) Note that the Java programming language deprecated `Object.finalize()` in Java 9, and deprecated it for removal in Java 18. Java's alternative to `Object.finalize()` is `PhantomReference` and its convenient wrapper `Cleaner`. It behaves very similar to Ruby's `ObjectSpace::define_finalizer` in that `PhantomReference` does not have access to the dead object, either, and avoids the infamous "object resurrection" problem in Java's `Object.finalize()`. # If we have to use `dfree` If we have to use `dfree` to finalize a TypedData instance, it may be profitable to run it in a "worker" thread concurrently with the mutator. But it may not be profitable to run them in multiple "worker" threads in parallel. From our experience of integrating MMTk with CRuby (See [this paper](https://wks.github.io/downloads/pdf/ruby-ismm-2025.pdf)), (1) finalization in the form of `obj_free` takes up the majority of the GC time when using MMTk, and (2) running `obj_free` in multiple parallel threads slows down the execution of `obj_free` instead of speeding it up because the `free()` function in libc does not scale when called in parallel. Since `dfree` usually calls the `free()` function in libc, it may happen that parallelizing the execution of `obj_free` will make it even slower. `RUBY_TYPED_PARALLEL_FREE_SAFE` probably doesn't profit MMTk directly. In MMTk, most GC algorithms, including high-performance algorithms like Immix and StickyImmix, don't do lazy sweeping. (MarkSweep supports lazy sweeping, but it has inferior performance.) They make the memory of the dead objects ready for allocation immediately after GC. This means we need to call `obj_free` during GC in a weak reference processing stage (which is after determining the objects' life and death, but before reclaiming their space). We are currently working hard reducing the number of objects that need `obj_free` because in CRuby almost all types are subject to `obj_free` (at least in some states). It is possible to replicate what Java used to do, i.e. retaining an unreachable object after GC so that a dedicated finalizer thread can finalize them. This deprecated feature is quite similar to this proposal, i.e. a concurrent "worker" thread that sweeps object. MMTk is able to retain dead objects and postpone the finalization after GC this way. But the problem is still that the space of those objects cannot be reclaimed (and therefore used for allocation) until their `dfree` functions are called. If we retain too many dead objects, the GC will not free up enough memory to keep up with the pace of allocation. And we will be in an awkward situation where a mutator triggered GC, but the GC cannot reclaim more memory until the finalizer thread (which is another mutator) finalizes more objects and stop retaining objects. It may be worth annotation not just whether a TypedData is "safe to be freed concurrently with the mutator", but also whether "it is profitable to free it in parallel to other objects". For this to be true, the TypedData instance should not hold `malloc()`-ed memory that needs to be `free()`-ed. But that's too much implementation details and I don't think programmers can easily reason about it. # About parallel marking As Peter said, MMTk is using parallel marking (multiple GC workers marking objects in parallel). It should just work unless the TypedData does something anything in `dmark`, which it shouldn't anyway. `dmark` should just visit all fields that hold references. If the TypedData uses `RUBY_TYPED_DECL_MARKING`, it will have even less interference with the GC. Marking can profit greatly from parallelization. I think it should be easy for the default GC to parallelize marking. I added an attachment that shows the timeline of a GC in MMTK when running the railsbench using the (non-generational) Immix GC. (Note that this is using the MMTk GC module in https://github.com/ruby/mmtk/ and it doesn't include optimizations mentioned in [this paper](https://wks.github.io/downloads/pdf/ruby-ismm-2025.pdf) such as making `obj_free` unnecessary for T_STRING, T_ARRAY and T_MATCH. So finalization (calling `obj_free`) still dominates.) I am currently working on using eBPF to visualize the default GC, too. From my current observation (see attachment), there are several long marking events during early execution. But as the program makes progress, sweeping gradually starts to dominate. I am curious what the timeline would look like if we have concurrent sweeping. ---------------------------------------- Feature #22067: New TypedData bit to allow the type to be freed in parallel https://bugs.ruby-lang.org/issues/22067#change-117304 * Author: luke-gru (Luke Gruber) * Status: Open ---------------------------------------- CRuby `TypedData` types are used internally in the VM and in C extensions and currently any `free` functions for these types are run with the VM lock held as well as the VM barrier (or, in the case of MMTk, are not freed in parallel). In short, no other Ruby threads or GC threads can run while these `free` functions are called. In order to allow one or more worker GC threads to free these `TypedData` objects, we need a way to specify that a `TypedData` type can be freed in parallel alongside the Ruby GC thread or Ruby code that is being run by another thread. Otherwise, we cannot free these types in the workers and must rely on the Ruby GC thread to do so. This is because it can be unsafe to call these `free` functions, depending on how they're implemented. Most `TypedData` are safe to free in parallel. The exceptions are those `free` functions that read or modify global state without locking. ### Examples ```c static void example_data_free(void *ptr) { st_delete(live_example_datas, (st_data_t*)&ptr, NULL); // Not thread-safe! } ``` If 2 of these `TypedData` objects are freed at the same time, this could corrupt the `st_table`. A lock must be held when manipulating this table. Because Ruby code can also run alongside a sweep worker, the lock must also be held when adding to or iterating this table. ```c rb_nativethread_lock_t example_data_lock; static void example_data_free(void *ptr) { rb_native_mutex_lock(&example_data_lock); st_delete(live_example_datas, (st_data_t*)&ptr, NULL); rb_native_mutex_unlock(&example_data_lock); } static void live_examples_add(void *ptr) { rb_native_mutex_lock(&example_data_lock); st_insert(live_example_datas, (st_data_t)ptr, (st_data_t)ptr); rb_native_mutex_unlock(&example_data_lock); } static void Init_example(void) { rb_native_mutex_initialize(&example_data_lock); } ``` If a CRuby developer or user wants to make their code compaction-safe, they don���t need to worry about parallel sweep workers because the workers don't run during compaction. ### Proposal I propose adding a new flag to `TypedData` that allows both CRuby developers and extension authors to opt-in to allow their `TypedData` type to be freed in parallel. It could be called something like `RUBY_TYPED_PARALLEL_FREE_SAFE`. In our branch where we are developing parallel sweeping, most `TypedData` internal to the VM are given this flag. I believe C extension authors would opt in as well if they see that parallel sweeping gives good performance benefits to Ruby applications. We would need to document what is safe and what is not safe inside these `free` functions for types that are marked with this bit. If the user needs to use a native mutex to protect their `TypedData` from being corrupted when freed in parallel, they must do so. However, if they lock this native mutex in non-free function code paths as well, they may not allocate objects or use `ruby_xmalloc` while this mutex is held. These are some examples of trickiness when it comes to concurrency in the Ruby VM. I believe having a section in the extension documentation about these types of free functions (or elsewhere such as the Concurrency Guide) would give CRuby developers and extension authors more confidence in adding this bit to their types. ### A possible future: parallel marking? If in the future CRuby gets parallel marking, we believe we probably would need another bit for `TypedData` so we can register them as parallel-mark safe. If that���s the case, it���s unfortunate that authors that want this feature would need to update their extension again with a new bit if this bit were to be made public. One could argue that we should have a single bit that indicates safety for both parallel freeing and parallel marking. However, we believe the specifics of what could and couldn���t be executed inside these `free` and `mark` functions would be too hard to work out today for a combined bit without locking us into a specific parallel marking implementation. ### Alternative We could not expose the parallel-free-safe bit to extension authors and only free internal `TypedData` objects in parallel. However, this does slow the current implementation of parallel sweeping down because even if one of these objects is on a heap page, the Ruby GC thread needs to further post-process the page after the sweep thread sweeps it. It would also limit further optimizations to parallel sweeping. ### Details of the bit Worker threads could only free `TypedData` objects that have this new bit set alongside the `RUBY_TYPED_FREE_IMMEDIATELY` bit. Otherwise, the Ruby GC thread must free them. Also, if the type has `0` or `RUBY_TYPED_DEFAULT_FREE`/`RUBY_DEFAULT_FREE` as the free function, it can be freed in parallel. ### Conclusion Parallel sweeping is being actively developed and adding this bit increases performance of the developing implementation of parallel sweeping. It allows `TypedData` types to be freed by worker thread(s), which decreases GC pause times. Since `TypedData` are also exposed to extension authors, we need to whitelist these types for parallel freeing, as not all of these functions are currently safe. By documenting what is and isn���t safe, extension authors can write correct `free` functions that are safe to be freed by worker threads. Alternatively, we could have this bit be internal to CRuby. ---Files-------------------------------- Screenshot_20260513_185452.webp (99.8 KB) Screenshot_20260513_192449.webp (31.9 KB) Screenshot_20260513_192734.webp (22.4 KB) Screenshot_20260513_192910.webp (27.3 KB) -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/