[ruby-core:101298] [Ruby master Feature#17375] Add scheduler callbacks for transferring fibers

Issue #17375 has been reported by nevans (Nicholas Evans).

----------------------------------------
Feature #17375: Add scheduler callbacks for transferring fibers
https://bugs.ruby-lang.org/issues/17375

* Author: nevans (Nicholas Evans)
* Status: Open
* Priority: Normal
----------------------------------------
When working on #17325 (`Fiber#cancel`) and #17331 (`Fiber#raise` on transferring fibers), two very reasonable questions keep coming up:
* "how and when should control pass back to the current fiber?" and
* "is it expected that terminating fibers will return to the root fiber chain?"

Rather than deal with that complexity, I've passed the buck: that's out of scope for those tickets. The end user should just call their scheduler library and let it coordinate, right?

But with a couple of optional hooks on the scheduler, I think we can answer both of those questions.  I'm not sure that these are the ideal API, but what do you think about *something* similar to the following?

### "how and when should control pass back to the current fiber?"

```ruby
# Called before transferring into another fiber.
# @param target [Fiber] the fiber that will be transferred into
# @param reason [:transfer, :raise, :cancel] How the transfer was initiated 
# @param args the arguments sent to transfer, raise, or cancel.
#
# @raise [FiberTransferInterrupted] to prevent the transfer without raising an
#     exception in the calling fiber.
# @raise [Exception] prevents the transfer and raises in the calling fiber
#
# This can be used to ensure the current fiber is appropriately scheduled for
# return, and it can also prevent the transfer or schedule the transfer to
# happen asynchronously.
#
# In addition to raising exceptions, any call to a fiber switching method (e.g.
# resume, yield, or transfer) will prevent the transfer. When a transfer is
# prevented, any associated cancellation or exception will not happen.
#
# This will only be called for transfers, not for resume, yield, or termination.
def transferring(target, reason, *args)
  # one possible implementation:
  return if Fiber.current == @scheduler # always allow transfer from scheduler
  if target == @scheduler
    # guard transfer to the scheduler
    raise FiberError, "invalid transfer to scheduler" if invalid?(reason, *args)
  else
    # schedule all transfers instead of running immediately
    @next_tick << [target, reason, *args]
    @next_tick << [Fiber.current, :transfer] unless blocking?
    @scheduler.transfer
  end
end
```

This would be useful for more than just `Fiber#raise` and `Fiber#cancel`. It could allows any non-scheduler code to safely call `Fiber#transfer` (or to indirectly transfer via `#raise` or `#cancel`) without confusing or breaking the scheduler. Or the scheduler could **disallow** any transfers but its own. Or it could intercept certain internal framework exceptions. It allows the scheduler some control over transfer and over raise/cancel with transferring fibers.

### "exceptions raised from terminating transferred fibers will return to the root fiber chain"

```ruby
# Select the return fiber for a transferring fiber when it terminates.
# @param terminating [Fiber] The terminating fiber
# @param retval [Object] return value of the terminating fiber
# @param error [Exception, nil] raised by terminating fiber
# @return [Fiber, nil] fiber to transfer into. `nil` uses default behavior
#
# If the selected return fiber can't be transferred to (because it is yielding
# or resuming or dead), FiberError will be raised on root fiber chain.
#
# This will be run in the terminating fiber after its block has completed.
# If this raises an exception, that will be raised on the root fiber chain.
#
def select_return_fiber(terminating, retval, error)
  supervisor = @supervised[terminating]
  if supervisor&.alive?
    supervisor
  elsif @scheduled.include?(terminating)
    @scheduler
  elsif !error
    raise FiberError, "unsupervised transfer fiber terminated"
  end
end
```

In addition to answering the question raised by #17325 and #17331, I think this also simplifies some other useful patterns, e.g. supervisors.

It would also let me easily fix one of the things that ruby 3.0 breaks in my current code: I liked to "init" my transfer fibers by first resuming into them from their supervisor and then immediately transferring back out. That sets the return_fiber for when it terminates. A workaround is to use an ensured `supervisor.transfer` on the last line of the fiber and then abandon the almost dead fiber. But that might lead to a bug later if some other code held onto a reference to that fiber, saw it was still alive, and transferred into it (unlikely, but plausible). And it's still brittle: if any errant code calls `Fiber.yield`, the return_fiber will be lost and can never be recovered. Letting the scheduler manage this would provide the lost ruby 2 functionality and more.

What do you think?



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>