From: nicholas.evans@... Date: 2020-12-07T23:31:23+00:00 Subject: [ruby-core:101298] [Ruby master Feature#17375] Add scheduler callbacks for transferring fibers Issue #17375 has been reported by nevans (Nicholas Evans). ---------------------------------------- Feature #17375: Add scheduler callbacks for transferring fibers https://bugs.ruby-lang.org/issues/17375 * Author: nevans (Nicholas Evans) * Status: Open * Priority: Normal ---------------------------------------- When working on #17325 (`Fiber#cancel`) and #17331 (`Fiber#raise` on transferring fibers), two very reasonable questions keep coming up: * "how and when should control pass back to the current fiber?" and * "is it expected that terminating fibers will return to the root fiber chain?" Rather than deal with that complexity, I've passed the buck: that's out of scope for those tickets. The end user should just call their scheduler library and let it coordinate, right? But with a couple of optional hooks on the scheduler, I think we can answer both of those questions. I'm not sure that these are the ideal API, but what do you think about *something* similar to the following? ### "how and when should control pass back to the current fiber?" ```ruby # Called before transferring into another fiber. # @param target [Fiber] the fiber that will be transferred into # @param reason [:transfer, :raise, :cancel] How the transfer was initiated # @param args the arguments sent to transfer, raise, or cancel. # # @raise [FiberTransferInterrupted] to prevent the transfer without raising an # exception in the calling fiber. # @raise [Exception] prevents the transfer and raises in the calling fiber # # This can be used to ensure the current fiber is appropriately scheduled for # return, and it can also prevent the transfer or schedule the transfer to # happen asynchronously. # # In addition to raising exceptions, any call to a fiber switching method (e.g. # resume, yield, or transfer) will prevent the transfer. When a transfer is # prevented, any associated cancellation or exception will not happen. # # This will only be called for transfers, not for resume, yield, or termination. def transferring(target, reason, *args) # one possible implementation: return if Fiber.current == @scheduler # always allow transfer from scheduler if target == @scheduler # guard transfer to the scheduler raise FiberError, "invalid transfer to scheduler" if invalid?(reason, *args) else # schedule all transfers instead of running immediately @next_tick << [target, reason, *args] @next_tick << [Fiber.current, :transfer] unless blocking? @scheduler.transfer end end ``` This would be useful for more than just `Fiber#raise` and `Fiber#cancel`. It could allows any non-scheduler code to safely call `Fiber#transfer` (or to indirectly transfer via `#raise` or `#cancel`) without confusing or breaking the scheduler. Or the scheduler could **disallow** any transfers but its own. Or it could intercept certain internal framework exceptions. It allows the scheduler some control over transfer and over raise/cancel with transferring fibers. ### "exceptions raised from terminating transferred fibers will return to the root fiber chain" ```ruby # Select the return fiber for a transferring fiber when it terminates. # @param terminating [Fiber] The terminating fiber # @param retval [Object] return value of the terminating fiber # @param error [Exception, nil] raised by terminating fiber # @return [Fiber, nil] fiber to transfer into. `nil` uses default behavior # # If the selected return fiber can't be transferred to (because it is yielding # or resuming or dead), FiberError will be raised on root fiber chain. # # This will be run in the terminating fiber after its block has completed. # If this raises an exception, that will be raised on the root fiber chain. # def select_return_fiber(terminating, retval, error) supervisor = @supervised[terminating] if supervisor&.alive? supervisor elsif @scheduled.include?(terminating) @scheduler elsif !error raise FiberError, "unsupervised transfer fiber terminated" end end ``` In addition to answering the question raised by #17325 and #17331, I think this also simplifies some other useful patterns, e.g. supervisors. It would also let me easily fix one of the things that ruby 3.0 breaks in my current code: I liked to "init" my transfer fibers by first resuming into them from their supervisor and then immediately transferring back out. That sets the return_fiber for when it terminates. A workaround is to use an ensured `supervisor.transfer` on the last line of the fiber and then abandon the almost dead fiber. But that might lead to a bug later if some other code held onto a reference to that fiber, saw it was still alive, and transferred into it (unlikely, but plausible). And it's still brittle: if any errant code calls `Fiber.yield`, the return_fiber will be lost and can never be recovered. Letting the scheduler manage this would provide the lost ruby 2 functionality and more. What do you think? -- https://bugs.ruby-lang.org/ Unsubscribe: