[ruby-core:104544] [Ruby master Feature#18020] Introduce `IO::Buffer` for fiber scheduler.

Issue #18020 has been updated by ioquatix (Samuel Williams).


@eregon thanks for your discussion.

Something like this is required for the fiber scheduler interface. It's also required for efficient IO. Many people have asked for this feature, maybe there is something I don't know but why they didn't use `FFI::Pointer` and why is there interest in `IO::Buffer` from other people? If they could already use `FFI::Pointer`, why didn't they?

I'm not against `FFI::Pointer` but there are probably some subtle differences in that I'm initially interested in the IO layer and zero-copy IO. I'm not sure how efficiently `FFI::Pointer` is implemented either, but this will be something we can map directly to our use case which is specifically IO related. Network IO does have specific requirements around efficient decode of binary data.

> It might be more valuable to make ffi a default or bundled gem, which also brings much more capabilities. ffi is already a bundled gem for both JRuby and TruffleRuby.

This may be a problem as the fiber scheduler is part of the core interface. So, whatever we have, it must be part of Ruby core? I'm pretty keen to keep the definition of the fiber scheduler as simple as possible, so introducing a relatively straight forward memory buffer is probably preferable to pulling in all of `ffi`, at least from a complexity PoV.

----------------------------------------
Feature #18020: Introduce `IO::Buffer` for fiber scheduler.
https://bugs.ruby-lang.org/issues/18020#change-92822

* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
After continuing to build out the fiber scheduler interface and the specific hooks required for `io_uring`, I found some trouble within the implementation of `IO`.

I found that in some cases, we need to read into the `rb_io_buffer_t` struct directly. I tried creating a "fake string" in order to transit back into the Ruby fiber scheduler interface and this did work, but I was told we cannot expose fake string to Ruby code.

So, after this, and many other frustrations with using `String` as a IO buffer, I decided to implement a low level `IO::Buffer` based on my needs for high performance IO, and as part of the fiber scheduler interface. Going forward, this can form the basis of newer interfaces like `IO::Buffer#splice` and so on. We can also add support for `IO#read(n, buffer)` rather than string. This avoids many encoding and alignment issues.

While I'm less interested in the user facing interface at this time, I believe we can introduce it incrementally. Initially my focus is on the interface requirements for the fiber scheduler. Then, I'll look at how we can integrate it more into `IO` directly. The goal is to have this in place for Ruby 3.1.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>