From: "Eregon (Benoit Daloze) via ruby-core" Date: 2026-09-11T20:58:29+00:00 Subject: [ruby-core:126665] [Ruby Feature#22274] Make `IO::Buffer` no longer experimental. Issue #22274 has been updated by Eregon (Benoit Daloze). (apologies for the reply partly written by AI, the base argument is mine, and I found a problematic case with raw addresses for slices) Right, that "every buffer is a slice of an underlying allocation" model is exactly the right framing, it's essentially the offset-based design (a slice over an allocation whose identity is stable even if its backing storage moves), and it's what TruffleRuby and JRuby would naturally implement. I have thought more about it and I believe a slice should stay valid across a reallocation that moves the source, and that CRuby itself would be better off with offset-based slices, not just leaving it as "implementation-defined" (which is confusing for users by having less clear/undefined semantics). It's an easy change on the C side, because the validation path already does the work. `io_buffer_validate_slice` fetches the source's current base and size on every access (`RSTRING_GETMEM` / `rb_io_buffer_get_bytes`) and literally computes `offset = slice_base - source_base` to bounds-check. So the absolute `base` stored in the slice is redundant with `source_base + offset`, the offset is re-derived on every single access anyway. If a slice instead stored that `offset` (relative to the root source) and resolved `base = source_base + offset` at access/lock time, with the same `offset + length <= source_size` check, then a `resize` that relocates the source (`realloc`, or `mremap(..., MREMAP_MAYMOVE)` for mapped buffers) keeps the slice valid as long as its range still fits (the most reliable behavior), at the same per-access cost (same source fetch, same arithmetic, just `source_base + offset` instead of `slice_base - source_base`). The more important reason, though, is that address-based validation is silently wrong in a case that offset-based can't hit. Because a slice keeps a fixed *absolute* pointer and re-derives its offset from the source's current base, any reallocation that lands *overlapping but shifted* from the old one silently rebinds the slice to a different logical region: - Source at 1000, `slice(16, 16)` stores absolute base 1016. - Source is freed and reallocated at 1006 (size 64), overlapping the old range. - Now `offset = 1016 - 1006 = 26`: the slice reports `valid? == true` but refers to logical offset **26**, not 16. So it's not just the exact-same-address regrow (which preserves the logical offset and is relatively benign), it's *any* overlapping reallocation, and the offset drifts by the shift amount. This stays in-bounds of the live allocation, so it's memory-safe (no OOB, no crash), but it means `get_string` reads the wrong region and, worse, `set_string`/`copy` through the slice silently scribble over the wrong logical bytes of live data, all while `valid?` is true. An offset-based slice is immune by construction: the logical offset is fixed at creation and never re-derived from a possibly-shifted base, so a slice can only ever refer to the region it was created for, or be invalid: it can't rebind. So rather than leaving invalidation unspecified, I'd suggest the observable contract be: a slice tracks a logical `[offset, length)` range of its source; while valid it sees exactly those bytes; it becomes invalid (raising `InvalidatedError`) only when the source is freed or shrunk past it. That's implementable identically on CRuby and JVM implementations, it survives relocation, and it removes the silent shifted-rebind that the raw-pointer design allows. The "stable address for syscalls" property is unaffected: we still resolve to a concrete `base` under the lock before handing it to a syscall / io_uring, and the source can't move while locked. Offset-based only changes behavior *across* resize, which is exactly the case the raw-pointer design gets wrong. The current docs describe the address-reuse revalidation as intended, commits `cc706f3956` and `4d5ae42629`, which is the part I'd revisit. ---------------------------------------- Feature #22274: Make `IO::Buffer` no longer experimental. https://bugs.ruby-lang.org/issues/22274#change-118959 * Author: ioquatix (Samuel Williams) * Status: Open * Assignee: ioquatix (Samuel Williams) ---------------------------------------- `IO::Buffer` was introduced in Ruby 3.1 by Feature #18020 as an experimental API. It provides an efficient buffer abstraction for fiber scheduler I/O, zero-copy operations, binary protocol implementations, and access from native extensions. Since then, the API has seen several Ruby releases of real-world use and substantial development. Ruby 4.1 now has cohesive semantics for buffer ownership and lifecycle, slicing, locking, string and file mappings, MemoryView integration, and single-transfer I/O operations. I propose making both the Ruby and public C interfaces of `IO::Buffer` non-experimental in Ruby 4.1. This would involve: * Removing the allocation-time experimental warning. * Removing the experimental status from the class documentation. * Removing `RB_IO_BUFFER_EXPERIMENTAL` from the public C header. * Removing warning suppression that is only required because `IO::Buffer` is experimental. * Updating NEWS to describe `IO::Buffer` as stable. `RUBY_IO_BUFFER_VERSION` would remain available for compile-time feature detection as the interface continues to evolve. Before stabilizing the interface, I would particularly appreciate feedback from JRuby and TruffleRuby maintainers. Related work: * Original proposal: https://bugs.ruby-lang.org/issues/18020 * CRuby implementation PR: https://github.com/ruby/ruby/pull/18486 * TruffleRuby implementation: https://github.com/truffleruby/truffleruby/pull/4248 * JRuby implementation: https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyIOBuffer.java * JRuby file-mapping issue: https://github.com/jruby/jruby/issues/8714 -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/