From: "Eregon (Benoit Daloze) via ruby-core" Date: 2026-09-09T22:22:34+00:00 Subject: [ruby-core:126624] [Ruby Feature#22274] Make `IO::Buffer` no longer experimental. Issue #22274 has been updated by Eregon (Benoit Daloze). Thank you for creating this issue to discuss it. I have several points in no particular order. * There is a [lot of work](https://github.com/ruby/ruby/pulls?q=is%3Apr+IO%3A%3ABuffer) on `IO::Buffer` recently, this is great. But it might also mean some new bugs, maybe we should wait some time so it gets "battle tested" (= in a release 4.1.0, and some time for it to be tested in production for various apps) before declaring it stable? * https://github.com/ruby/ruby/pull/18483 is a large breaking change, it swaps the order of arguments from `length, offset` to `offset, length`. That I think could break many existing usages of IO::Buffer in subtle ways. Which makes me think maybe there is a possibility this needs to be reverted, in which case having it marked stable looks a bit strange. I do hope it doesn't need to be reverted though, or at least if it is that it happens before the 4.1.0 release. I think we have never seen a core API changing positional arguments order before (or at least not before several deprecation/migration phases in between), but since `IO::Buffer` has been experimental so far maybe it's OK? * From my implementation of IO::Buffer in TruffleRuby I recall two things: `#slice` returns a "sliced IO::Buffer" which is pretty tricky to implement as e.g. just copying the `char*` in the slice is not OK, as the parent buffer might be resized, free'd, etc. So I think every slice needs to always go through the parent buffer, and can't assume the parent buffer didn't change. Also slices are writable, which limits a lot of that can be implemented. Currently CRuby does not go through the parent but captures the raw pointer and validates it, which affects semantics, see below. * Error messages seems inconsistent with other core API, e.g. `ArgumentError: Size can't be negative!` vs `ArgumentError: negative array size`. I think consistency is valuable there (otherwise I think it won't "feel" like a core API). I asked Claude to compare the TruffleRuby and CRuby master implementations and find potential issues or behavior that could be clarified/improved: --- I went through the current `master` implementation while checking what TruffleRuby needs to match for stabilization, and ran a differential + concurrency probe against a fresh `master` build (`4.1.0dev 2026-09-08`). A few things seem worth resolving before freezing the interface. ## 1. `set_string` is not lock-protected and can segfault under concurrent `resize` `IO::Buffer#set_string` with a payload ��� `IO_BUFFER_BLOCKING_SIZE` (1 MiB) segfaults on `master` when another thread resizes the same buffer concurrently: ```ruby Warning[:experimental] = false big = 4 * 1024 * 1024 60.times do b = IO::Buffer.new(big) src = "Z" * (big / 2) t = Thread.new { sleep(rand * 0.0008); b.resize(64 * 1024) rescue nil } b.set_string(src) # [BUG] Segmentation fault in io_buffer_memmove_blocking t.join end ``` ``` -- C level backtrace ------------------ __memcpy_avx_unaligned_erms io_buffer_memmove_blocking io_buffer.c:3011 rb_nogvl thread.c:1820 io_buffer_copy_from io_buffer.c:3063 io_buffer_set_string io_buffer.c:3312 ``` The cause is an asymmetry with `copy`. Both go through `io_buffer_copy_from` ��� `io_buffer_memmove`, which **releases the GVL** for copies ��� 1 MiB. `copy` was hardened to lock both buffers (`rb_io_buffer_locked_for_reading` / `_for_writing`, #18489), so a concurrent `resize` hits a locked buffer and cannot `realloc`. `set_string` (`io_buffer_set_string` ��� `io_buffer_copy_from`) takes **no** such lock, so the resize reallocates the base out from under the in-flight `memmove` ��� use-after-free. Running each write path against the same race in isolation: | method | result | why | |--------------|-------------|-----| | `set_string` | **SIGSEGV** | releases GVL, does not lock destination | | `copy` | ok | locks both sides | | `clear` | ok | keeps GVL (plain `memset`) | | `read` | ok | locks via `io_buffer_blocking_region` | The fix is presumably to give `set_string` the same locked scope `copy` already uses. ## 2. Slice validity is address-based ��� two asymmetric edge cases worth ratifying A slice captures an absolute pointer (`source->base + offset`) at creation and re-validates on every access by checking the pointer still falls within the source's current range (`io_buffer_validate_slice`). This correctly catches a freed/transferred source (base ��� `NULL` ��� `InvalidatedError`) and a source shrunk past the slice. But because validation is address-containment rather than offset-based, two cases are asymmetric, and I'd suggest confirming they're intended before the interface is frozen: 1. A resize that **moves** the allocation (`realloc`/`mremap` relocating) invalidates the slice **even when `offset + length` would still logically fit**. Reproduced for both `realloc` (grow to 1 MiB) and `mremap` (mapped buffer grown ~8 MiB): slice `valid?` becomes `false` every time. 2. If the source is later reallocated back over the slice's old address (ABA), the slice silently becomes **valid again but now points at semantically-unrelated bytes**. Reproduced: `free` a 64-byte buffer, `resize(64)` back ��� the stale slice resurrected and read the new contents in 8/10 runs. Memory-safe (still inside a live allocation), but returns wrong data. Case 2 is currently documented as intended (cc706f3956, 4d5ae42629). It's sharp enough to be worth an explicit decision, and relevant to this ticket's request for feedback from other implementations: **an offset-based slice design (the natural fit for TruffleRuby and JRuby, where the backing store can move under a managed GC) cannot reproduce the address-reuse revalidation semantics at all**, and would keep case 1's slices valid rather than invalidating them. If the spec is defined in terms of observable `valid?` results across resize, we'd want it worded so an offset-based implementation can conform. ## 3. Minor points - **`freeze` only guards lifecycle, not contents.** A frozen `IO::Buffer` still accepts `set_value` / `set_string` (only `free` / `resize` / `transfer` raise `FrozenError`). This is surprising for a frozen object and worth either documenting explicitly or reconsidering before stable. - **`initialize` is re-callable and leaks.** `io_buffer_initialize` overwrites `base`/`size`/`flags` without releasing the previous allocation. Re-invoking it on a mapped buffer silently leaks the mapping (`mapped?` flips to `false`, no `munmap`); on an internal buffer it leaks the old `malloc`. Doing it inside `#locked` also zeroes `lock_count`, so the ensuing unlock underflows and raises the misleading `LockedError: "Buffer not locked!"`. It'd be safer for `initialize` to release the old buffer first, or to refuse re-initialization. Happy to file (1) as a separate bug if that's easier to track. --- I think semantically it would be best to use "parent buffer + offset" for slices on CRuby too (cleaner semantics, easier to understand and document, also harder to misuse). Absolute pointers are not available on JRuby at all, and on TruffleRuby for Ruby Strings living in the managed heap (`byte[]`). ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-118864 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyIOBuffer.java * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/