From: "kjtsanaktsidis (KJ Tsanaktsidis) via ruby-core" <ruby-core@...>
Date: 2024-01-22T05:29:23+00:00
Subject: [ruby-core:116362] [Ruby master Bug#20197] Postponed job invocations are significantly reduced in Ruby 3.3

Issue #20197 has been updated by kjtsanaktsidis (KJ Tsanaktsidis).


OK, I investigated this a bit. The reason your example only runs the callback once is that the "is-an-interrupt-pending?" flag is a per-thread (technically per execution context) state. Normally, the `rb_postponed_job_register_one` (and the new `rb_postponed_job_trigger` function as well) will set the interrupt-pending flag on the currently thread, which is what you want when running in a signal handler. However, when you're running from a non-ruby thread, you need to pick a ruby thread to set the flag on.

The previous implementation before https://github.com/ruby/ruby/commit/1f0304218cf00e05a4a126196676ba221ebf91f6 would pick `vm->ractor.main_ractor->threads.running_ec`, which is actually the previous thread to run if there is no currently running thread. Now, however, we always pick the main thread (this seems to be because the semantics of `running_ec` are different when using the new M:N threading model).

Your example has the main thread sleeping, so it will never check the interrupt-pending flag. Thus, now it doesn't run the callbacks (until the other thread is done).


This PR will I think fix the issue with your reproduction: https://github.com/ruby/ruby/pull/9633.

Are you able to share some more details about your use-case for postponed_job by the way? I ask, because even with this fix (and restoring the Ruby 3.2 behaviour), there are still some race conditions possible (e.g. `running_ec` might have _just_ entered a long sleep). If we want to make it possible to robustly run postponed jobs "soon" from non-Ruby threads, I think we would need:

* Add a VM-global interrupt flag
* Have `rb_thread_check_ints` check the VM-global flag as well as its own EC-local flag,
* Have `rb_postponed_job_trigger` see if we're running on a Ruby thread, and if not, set the VM-global interrupt flag instead of the EC-local one.

This seems like a fair bit of extra complexity to carry around though if there isn't a use-case which requires it.

----------------------------------------
Bug #20197: Postponed job invocations are significantly reduced in Ruby 3.3
https://bugs.ruby-lang.org/issues/20197#change-106382

* Author: osyoyu (Daisuke Aritomo)
* Status: Open
* Priority: Normal
* Assignee: kjtsanaktsidis (KJ Tsanaktsidis)
* ruby -v: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-linux]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
The number of postponed job invocations has been significantly reduced in Ruby 3.3.
While my understanding is that postponed jobs provide no guarantee of how soon registered callbacks will fire, I believe the current rate is too low for practical usage, especially for profilers such as StackProf.

A git bisect led me to https://github.com/ruby/ruby/commit/1f0304218cf00e05a4a126196676ba221ebf91f6 which obviously seems to be related, but I'm not sure why.

## Repro

### Expected

The job fires (nearly) immediately after being triggered.

In the following example, the job is triggered every 100 ms.

```
% ruby bin/test.rb # runs for 3 seconds
count: 1
count: 2
(snip)
count: 29
```

### Actual

The job gets fired only once.

```
% ruby bin/test.rb
count: 1
```

### Code

```ruby
require 'mycext'

time = Time.now
th = Thread.new do
  loop do
    sleep 0.01
    break if Time.now - time > 3 # run for 3 seconds
  end
end
th.join
```

```c
#include <pthread.h>
#include <stdio.h>
#include <time.h>

#include "ruby.h"
#include "ruby/debug.h"

int called_count;

void
postponed_job(void *ptr)
{
  called_count++;
  printf("count: %d\n", called_count);
}

_Noreturn void *
pthread_main(void *_)
{
  while (1) {
    rb_postponed_job_register_one(0, postponed_job, NULL);

    // Sleep for 100 ms
    struct timespec ts;
    ts.tv_sec = 0;
    ts.tv_nsec = 100 * 1000 * 1000;
    nanosleep(&ts, NULL);
  }
}

RUBY_FUNC_EXPORTED void
Init_mycext(void)
{
  called_count = 0;

  pthread_t pthread;
  pthread_create(&pthread, NULL, pthread_main, NULL);
}
```



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/