From: ciconia@... Date: 2021-07-11T16:39:02+00:00 Subject: [ruby-core:104579] [Ruby master Bug#17664] Behavior of sockets changed in Ruby 3.0 to non-blocking Issue #17664 has been updated by ciconia (Sharon Rosner). > In my testing, comparing `io_uring` `io_read` and `io_write` operations, perform about 20% worse in practice in my benchmarks. This was surprising to me. My current understanding as to why it's slow is because when we perform `io_read`, internally it performs `read`, but because we have to defer that operation until the next iteration of the run loop, we pay quite a bit latency cost here. I think the increased latency is to be expected. Did you measure throughput? In my own benchmarks (on Polyphony) I've seen better throughput, slightly worse latency (don't remember the numbers). > In my benchmarks using 64+ connections, I only observed OP_READ -> EAGAIN < 20 times out of 1 million requests. My understanding is sockets are buffered, so normally you'll see EAGAIN only if you saturate them. > Using blocking sockets: Looking at those numbers a few things stand out (I'm assuming your benchmark is with some kind of HTTP server): - There's no line for `write` (is the blocking version working correctly?) - `fcntl` is at 17% - this is a serious cost if you need to do this on every `read`/`write` - `io_uring_enter` is called once for each `accept`, so apparently there's no batching of SQEs. Skimming your [io_uring backend code](https://github.com/socketry/event/blob/8c4449ebe0a3c76681655cf175d5aa6589934a9c/ext/event/backend/uring.c#L428-L454) I see you're iterating over available CQEs and resume the fiber for each CQE *while iterating*. In my experience you'll get better numbers if you put those resumable fibers in an array instead, then resume them one by one after having exhausted all available CQEs. Hope this helps! ---------------------------------------- Bug #17664: Behavior of sockets changed in Ruby 3.0 to non-blocking https://bugs.ruby-lang.org/issues/17664#change-92861 * Author: ciconia (Sharon Rosner) * Status: Assigned * Priority: Normal * Assignee: ioquatix (Samuel Williams) * ruby -v: 3.0.0 * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN ---------------------------------------- I'm not sure this is a bug, but apparently a change was introduced in Ruby 3.0 that makes sockets non-blocking by default. This change was apparently introduced as part of the work on the [FiberScheduler interface](https://github.com/ruby/ruby/blame/78f188524f551c97b1a7a44ae13514729f1a21c7/ext/socket/init.c#L411-L434). This change of behaviour is not discussed in the Ruby 3.0.0 release notes. This change complicates the implementation of an io_uring-based fiber scheduler, since io_uring SQE's on fd's with `O_NONBLOCK` can return `EAGAIN` just like normal syscalls. Using io_uring with non-blocking fd's defeats the whole purpose of using io_uring in the first place. A workaround I have put in place in the Polyphony [io_uring backend](https://github.com/digital-fabric/polyphony/blob/d3c9cf3ddc1f414387948fa40e5f6a24f68bf045/ext/polyphony/backend_io_uring.c#L28-L47) is to make sure `O_NONBLOCK` is not set before attempting I/O operations on any fd. -- https://bugs.ruby-lang.org/ Unsubscribe: