From: eregontp@... Date: 2021-07-08T15:29:44+00:00 Subject: [ruby-core:104545] [Ruby master Feature#18020] Introduce `IO::Buffer` for fiber scheduler. Issue #18020 has been updated by Eregon (Benoit Daloze). ioquatix (Samuel Williams) wrote in #note-5: > why is there interest in `IO::Buffer` from other people? I was not aware of that, did people specifically ask for IO::Buffer? > I'm not sure how efficiently `FFI::Pointer` is implemented either. It's very efficient, most likely as efficient or better than IO::Buffer. It's already intensified on CRuby, TruffleRuby and JRuby. Doing that work again for IO::Buffer feels redundant to me. > but this will be something we can map directly to our use case which is specifically IO related. You can build a FFI::Pointer around a raw address, so it's also possible to ensure it's aligned, etc. > Network IO does have specific requirements around efficient decode of binary data. FFI::Pointer has read/write/get/put_byte/short/int/long, etc, so I think that should cover it. > This may be a problem as the fiber scheduler is part of the core interface. So, whatever we have, it must be part of Ruby core? Not necessarily, nothing forces the scheduler interface to yield an IO::Buffer object, it could let the user choose whatever they want to use to read/write to a given native address. I think that's actually more flexible, and might avoid an extra IO::Buffer allocation which would otherwise not be needed. ---------------------------------------- Feature #18020: Introduce `IO::Buffer` for fiber scheduler. https://bugs.ruby-lang.org/issues/18020#change-92823 * Author: ioquatix (Samuel Williams) * Status: Open * Priority: Normal ---------------------------------------- After continuing to build out the fiber scheduler interface and the specific hooks required for `io_uring`, I found some trouble within the implementation of `IO`. I found that in some cases, we need to read into the `rb_io_buffer_t` struct directly. I tried creating a "fake string" in order to transit back into the Ruby fiber scheduler interface and this did work, but I was told we cannot expose fake string to Ruby code. So, after this, and many other frustrations with using `String` as a IO buffer, I decided to implement a low level `IO::Buffer` based on my needs for high performance IO, and as part of the fiber scheduler interface. Going forward, this can form the basis of newer interfaces like `IO::Buffer#splice` and so on. We can also add support for `IO#read(n, buffer)` rather than string. This avoids many encoding and alignment issues. While I'm less interested in the user facing interface at this time, I believe we can introduce it incrementally. Initially my focus is on the interface requirements for the fiber scheduler. Then, I'll look at how we can integrate it more into `IO` directly. The goal is to have this in place for Ruby 3.1. -- https://bugs.ruby-lang.org/ Unsubscribe: