From: "tenderlovemaking (Aaron Patterson) via ruby-core" Date: 2025-07-24T20:44:50+00:00 Subject: [ruby-core:122862] [Ruby Feature#21039] Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks Issue #21039 has been updated by tenderlovemaking (Aaron Patterson). Eregon (Benoit Daloze) wrote in #note-19: > tenderlovemaking (Aaron Patterson) wrote in #note-18: > > This can't be a serious suggestion? It's basically saying that no existing Sinatra code could run inside a Ractor based webserver. If we had new syntax for `Ractor.shareable_proc`, I could see that being easier to swallow, but this doesn't seem acceptable (to me at least). > > Yeah, I understand your concern, and I meant this mostly as a workaround, while finding what other parts of Ractor prevents using Ractor for realistic Ruby code. > OTOH I'm rather skeptical that even if `Ractor.shareable_proc` would be allowed with a non-literal block that we'd be able to run Sinatra (or Rails, etc) apps on Ractors. > Not even MSpec runs on Ractor, and that's pretty simple logic, yet making it Ractor-compatible in a non-ugly-and-complicated way seems very hard. I don't have any numbers, but my intuition is that most non-literal, global procs (procs that are reachable via the application), including ones provided by the user, don't depend on environment mutations that occur outside of the block. Outside of iterators, depending on mutations to one's captured environment would be extremely confusing and hard to track behavior, so IME most people don't do it in practice. But besides that, I'm just proposing that `Ractor.shareable_proc` be allowed to take a non-literal block. This would allow frameworks to pick and choose which lambdas should be "safe" for a Ractor. If someone had a Sinatra app that depended on env mutations like this: ```ruby counter = 0 get "/" do # assume the proc gets copied here so counter is 0 "Hello world #{counter}" counter += 1 end counter += 123 ``` I think a user running a Ractor webserver would report an issue with the webserver since it would behave "as expected" on a non-Ractor webserver. > I think it would be good to have a construct (be it syntax or a Kernel or Proc method), independent of Ractor, to create a Proc which snapshots its environment, and is not allowed to write to its environment. > That way, that Proc would have the same semantics whether Ractors are used or not. I'm not sure why it matters whether the proc can write to the captured env or not, since we can just copy the environment and attach it to the proc. To me this is similar to dup'ing an object. If I mutate one copy, I don't expect those mutations to be reflected in the other. > That concept on its own is very useful for optimizations and JITs, in fact TruffleRuby [already has this functionality internally](https://github.com/oracle/truffleruby/blob/e805fbbf231d0680cb262c5dfd2278efd923bdc3/src/main/java/org/truffleruby/extra/TruffleGraalNodes.java#L91-L99). > One complication though is it's pretty expensive to do this, as on every call to that method it allocates a new Proc and potentially copies + change the bytecode to replace captured variable reads with their values (thought there might be other ways to do this). > Having it as syntax or an intrinisified method (which would mean cannot be redefined and must be detectable at parse time, no metaprogramming call to it) would help. > It would be useful for `define_method` too, and would mean methods defined with `define_method` could be as fast as `def` when called. I've thought of doing this by mprotecting the escaped env and only allowing reads ���� > To make it useful for Ractor we'd need that construct to also support setting the receiver, as in the `Ractor.shareable_proc(self: 42) { }` case from above. > And it would also need to either check that captured variable values are shareable, or make them shareable. That part is a bit weird when not using Ractors though, especially making shareable. > Checking for shareable seems better anyway, because making them shareable would need to copy to be safe in general, but maybe the user want to do it inplace if they know that's safe, etc. > It could be something like: > ```ruby > captured = 7 > p = Proc.isolated(self: 6) { self * captured } > captured = 10 > p.call # => 42 > ``` > > Syntax seems better-defined for the semantics and probably would look cleaner, but I'm not sure what would be a good syntax, and then of course it can't work on older Ruby versions at all, even when not using Ractors. > Unless we use something cheeky like `proc { |a,b; isolated| }` maybe, but that wouldn't allow setting the self (would have to be done with `instance_exec` around it), and would have different semantics on different versions which is not great. Anyway, by not allowing non-literal blocks, we can't even abstract calls to `shareable_proc` and I think that really hampers the usefulness of Ractors. I really think we should find a way to support non-literal blocks. ---------------------------------------- Feature #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks https://bugs.ruby-lang.org/issues/21039#change-114154 * Author: Eregon (Benoit Daloze) * Status: Assigned * Assignee: ko1 (Koichi Sasada) ---------------------------------------- ```ruby def make_counter count = 0 nil.instance_exec do [-> { count }, -> { count += 1 }] end end get, increment = make_counter reader = Thread.new { sleep 0.01 loop do p get.call sleep 0.1 end } writer = Thread.new { loop do increment.call sleep 0.1 end } ractor_thread = Thread.new { sleep 1 Ractor.make_shareable(get) } sleep 2 ``` This prints: ``` 1 2 3 4 5 6 7 8 9 10 10 10 10 10 10 10 10 10 10 10 ``` But it should print 1..20, and indeed it does when commenting out the `Ractor.make_shareable(get)`. This shows a given block/Proc instance is concurrently broken by `Ractor.make_shareable`, IOW Ractor is breaking fundamental Ruby semantics of blocks and their captured/outer variables or "environment". It's expected that `Ractor.make_shareable` can `freeze` objects and that may cause some FrozenError, but here it's not a FrozenError, it's wrong/stale values being read. I think what should happen instead is that `Ractor.make_shareable` should create a new Proc and mutate that. However, if the Proc is inside some other object and not just directly the argument, that wouldn't work (like `Ractor.make_shareable([get])`). So I think one fix would to be to only accept Procs for `Ractor.make_shareable(obj, copy: true)`. FWIW that currently doesn't allow Procs, it gives `:828:in 'Ractor.make_shareable': allocator undefined for Proc (TypeError)`. It makes sense to use `copy` here since `make_shareable` effectively takes a copy/snapshot of the Proc's environment. I think the only other way, and I think it would be a far better way would be to not support making Procs shareable with `Ractor.make_shareable`. Instead it could be some new method like `isolated { ... }` or `Proc.isolated { ... }` or `Proc.snapshot_outer_variables { ... }` or so, only accepting a literal block (to avoid mutating/breaking an existing block), and that would snapshot outer variables (or require no outer variables like Ractor.new's block, or maybe even do `Ractor.make_shareable(copy: true)` on outer variables) and possibly also set `self` since that's anyway needed. That would make such blocks with different semantics explicit, which would fix the problem of breaking the intention of who wrote that block and whoever read that code, expecting normal Ruby block semantics, which includes seeing updated outer variables. Related: #21033 https://bugs.ruby-lang.org/issues/18243#note-5 Extracted from https://bugs.ruby-lang.org/issues/21033#note-14 -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/