From: daniel@...42.com Date: 2020-11-13T16:16:56+00:00 Subject: [ruby-core:100832] [Ruby master Feature#17298] Ractor's basket communication APIs Issue #17298 has been updated by Dan0042 (Daniel DeLorme). A comment about naming... my first impression was that a "basket" is a container with multiple objects to transmit between Ractors. Something like ```ruby basket = Ractor::Basket.new basket.to_move << a basket.to_copy << b basket.to_share << c Ractor.new(basket) do |basket| ... end ``` I found it counterintuitive to understand this was not the proposal. Since it only contains a single value, I think "envelope" is a better metaphor. An envelope usually contains a single letter, whereas a basket usually contains several (picnic) items. But from an OOP perspective I still don't understand why all those "basket" methods. Since there's already a Ractor::Basket object involved, why not send that via the existing methods? So rather than `Ractor.yield_basket Ractor.receive_basket` it feels more OO to have `Ractor.yield Basket.new(Ractor.receive)`. Actually in the case of the bridge ractor it would just be `Ractor.yield Ractor.receive`, and if the object being passed is a basket then it's efficient, otherwise it's not. ---------------------------------------- Feature #17298: Ractor's basket communication APIs https://bugs.ruby-lang.org/issues/17298#change-88474 * Author: ko1 (Koichi Sasada) * Status: Open * Priority: Normal ---------------------------------------- This ticket proposes `send_basket`/`receive_basket`, `yield_basket`/`take_basket` APIs to make effective and flexible bridge ractors. ## Background When we want to send an object as a message, we usually need to copy it. Copying is achieved according to marshal protocol, and the receiver loads it immediately. If we want to make a bridge ractor that receives a message and sends it to another ractor, immediate loading is not effective. ```ruby bridge = Ractor.new do Ractor.yield Ractor.receive end consumer = Ractor.new bridge do |from| obj = from.take do_task(obj) end msg = [1, 2, 3] bridge.send msg ``` In this case, the array (`[1, 2, 3]`) is * (1) dumped at the first `bridge.send msg` * (2) loaded at `Ractor.receive` * (3) dumped again at `Ractor.yield` * (4) loaded at `from.take` Essentially, we only need one dump/load pair, but now it needs two pairs. Mixing "moving" status is more complex. Now there is no way to pass the "moving" status to bridge ractors, so we cannot make a moving bridge. ## Proposal To make more effective and flexible bridge ractors, we propose new basket APIs * `Ractor.receive_basket` * `Ractor#send_basket` * `Ractor#take_basket` * `Ractor.yield_basket` They receive a message, retains the dumped state, and sends it without dumping again. We can rewrite the above example with these APIs. ```ruby bridge = Ractor.new do Ractor.yield_basket Ractor.receive_basket end consumer = Ractor.new bridge do |from| obj = from.take do_task(obj) end msg = [1, 2, 3] bridge.send msg ``` In this case, * (1) dumped at the first `bridge.send msg` * (2) loaded at `from.take` we only need one dump/load pair. ## Implementation https://github.com/ruby/ruby/pull/3725 ## Evaluation The following program makes four types of bridges and passes an array as a message through them. ```ruby USE_BASKET = false receive2yield = Ractor.new do loop do if USE_BASKET Ractor.yield_basket Ractor.receive_basket else Ractor.yield Ractor.receive end end end receive2send = Ractor.new receive2yield do |r| loop do if USE_BASKET r.send_basket Ractor.receive_basket else r.send Ractor.receive end end end take2yield = Ractor.new receive2yield do |from| loop do if USE_BASKET Ractor.yield_basket from.take_basket else Ractor.yield from.take end end end take2send = Ractor.new take2yield, Ractor.current do |from, to| loop do if USE_BASKET to.send_basket from.take_basket else to.send from.take end end end AN = 1_000 LN = 10_000 ary = Array.new(AN) # 1000 LN.times{ receive2send << ary Ractor.receive } # This program passes the message as: # main -> # receive2send -> # receive2yield -> # take2yield -> # take2send -> # main ``` The result is: ``` w/ basket API 0m2.056s w/o basket API 0m5.974s ``` on my machine (=~ x3 faster). (BTW, if we have a TVar, we can change the value `USE_BASKET` dynamically) ## Discussion ### Naming Of course, naming is an issue. Now, I named it "_basket" because the source code uses this terminology. There are other candidates: * container metaphor * package * parcel * box * envelope * packet (maybe bad idea because of confusion of networking) * bundle (maybe bad idea because of confusion of bin/bundle) * "don't touch the content" metaphor * raw * sealed * unopened I like "basket" because I like picnic. ### Feature Now, basket is represented by "Ractor::Basket" and there are no methods. We can add the following feature: * `Ractor::Basket#sender` returns the sending ractor. * `Ractor::Basket#sender = a_ractor` changes the sending ractor. * `Ractor::Basket#value` returns the content. There was another proposal `Ractor.recvfrom`, but we only need these APIs. -- https://bugs.ruby-lang.org/ Unsubscribe: