From: marcandre-ruby-core@...
Date: 2021-03-09T01:36:43+00:00
Subject: [ruby-core:102787] [Ruby master Bug#17679] Ractor incoming channel can consume unlimited resources

Issue #17679 has been updated by marcandre (Marc-Andre Lafortune).


It's not clear to me that this should be implemented at the Ractor level.

Both suggested approaches can be handled in Ruby, for example using an intermediary Ractor...

```ruby
DONE = Object.new.freeze

MIDDLEMAN = Ractor.new do
  on_queue = 0
  loop do
    message = Ractor.receive
    if message == DONE
      on_queue -= 1
    else
      if (on_queue > 32_000)
        puts "Too many requests, skipping #{message}"
        next
      end
      on_queue += 1
      DOER.send(message)
    end
  end
end

DOER = Ractor.new do
  loop do
    message = Ractor.receive
    sleep 0.01
    puts "Processed #{message}"
    MIDDLEMAN.send(DONE)
  end
end

counter = 0
while true
  counter += 1
  MIDDLEMAN.send(counter)
end
```

If/when a non-blocking `receive` will be available, the receiving ractor could also handle it's waiting queue internally

----------------------------------------
Bug #17679: Ractor incoming channel can consume unlimited resources
https://bugs.ruby-lang.org/issues/17679#change-90806

* Author: marcotc (Marco Costa)
* Status: Assigned
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* ruby -v: ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
## Background

In the [ddtrace](https://github.com/DataDog/dd-trace-rb) gem, we want to move telemetry trace sending to a separate background Ractor. We���re concerned that if something goes wrong/gets delayed in this background Ractor, more and more data will accumulate in the send/receive channel until the Ruby VM crashes because it runs out of memory.

## How to reproduce (Ruby version & script)

```ruby
receiver_ractor = Ractor.new do
  loop do
    message = Ractor.receive
    sleep 1
    puts "Processed #{message}"
  end
end

counter = 0
while true
  counter += 1
  receiver_ractor.send(counter)
end
```

## Expectation and result
The result is that the Ruby VM crashes due to out of memory.
We expect the Ruby VM to not crash.

## Suggested solutions
Some ideas on how this can be improved:
* Having a way for the sender of data to detect if the receiver Ractor is falling behind (approximate size of queue, timestamp of last processed item, or similar?).
* Having a way to limit the Ractor message receive buffer.





-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>