[ruby-core:100727] [Ruby master Feature#17307] A way to mark C extensions as thread-safe, Ractor-safe, or unsafe

Issue #17307 has been updated by Eregon (Benoit Daloze).


@ko1 discussed with @matz and they concluded with this interface:
```c
Init_foo()
{
  RB_EXT_CONFIG({
    .ractor_safe = true,
    .thread_safe = true,
  });
  rb_define_method(mod, "foo", func, arity);
}
```

Sounds good to me.
Here is a sketch of an implementation, also with a "3rd variant" to check it's extensible if we need it in the future:

```c
#include <stdbool.h>
#include <stdio.h>
struct rb_ext_config {
  bool ractor_safe;
  bool thread_safe;
  bool third_variant_safe;
};
#define RB_EXT_CONFIG(...) \
  struct rb_ext_config config = __VA_ARGS__; \
  rb_ext_config(&config);
void rb_ext_config(struct rb_ext_config *config) {
  printf("ractor_safe: %d\n", config->ractor_safe);
  printf("thread_safe: %d\n", config->thread_safe);
  printf("third_variant_safe: %d\n", config->third_variant_safe);
}
int main(int argc, char const *argv[]) {
  RB_EXT_CONFIG({
    .ractor_safe = true,
    .thread_safe = true,
  });
  return 0;
}
```
outputs:
```
ractor_safe: 1
thread_safe: 1
third_variant_safe: 0
```

For backward compatibility, it will also be needed to wrap if in `#ifdef RB_EXT_CONFIG`, which seems fine.

However, this struct design means all implementations will have to define both the `ractor_safe` and `thread_safe` members,
otherwise it will fail to compile.

And if/when we would add a third member, then we would need some way to check if that member is declared, otherwise it wouldn't be backward compatible.
This seems to work, although I'm not sure if such usages of the preprocessor are well defined:
```c
  RB_EXT_CONFIG({
    .ractor_safe = true,
    .thread_safe = true,
#ifdef RB_EXT_CONFIG_VARIANT3
    .third_variant_safe = true,
#endif
  });
```

----------------------------------------
Feature #17307: A way to mark C extensions as thread-safe, Ractor-safe, or unsafe
https://bugs.ruby-lang.org/issues/17307#change-88370

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
----------------------------------------
I would like to design a way to mark C extensions as thread-safe, Ractor-safe, or unsafe (= needs process-global lock).
By default, if not marked, C extensions would be treated as unsafe for compatibility.

Specifically, TruffleRuby supports C extensions, but for scalability it is important to run at least some of them in parallel (e.g., HTTP parsing in Puma).
This was notably mentioned in my [RubyKaigi talk](https://speakerdeck.com/eregon/running-rack-and-rails-faster-with-truffleruby?slide=17).
TruffleRuby defaults to acquire a global lock when executing C extension code for maximum compatibility (Ruby code OTOH can always run in parallel).
There is a command-line option for that lock and it can be disabled, but then it is disabled for all C extensions.
The important property for TruffleRuby is that the C extension does not need a global lock, i.e., that it synchronizes any mutable state in C that could be accessed by multiple threads, such as global C variables.
I believe many C extensions are already thread-safe, or can easily become thread-safe, because they do not rely on global state and do not share the RData objects between threads.

Ractor also needs a way to mark C extensions, to know if it's OK to use the C extension in multiple Ractors in parallel, and that the C extension will not leak non-shareable objects from one Ractor to another, which would lead to bugs & segfaults.
Otherwise, C extensions could only be used on the main/initial Ractor (or need to acquire a process-global lock whenever executing C extension code and ensure no non-shareable objects leak between Ractors), which would be a very big limitation (almost every non-trivial application depends on a C extension transitively).

In both cases, global state in the C extension needs synchronization.
In the thread-safe case, mutable state in C that could be accessed by multiple Ruby threads needs to be synchronized too (there might be no such state, e.g., if C extension objects are created per Thread).
In the Ractor case, the C extension must never pass an object from a Ractor to another, unless it is a shareable object.

What do you think would be a good way to "mark" C extensions?
Maybe defining a symbol in the C extension, similar to the `Init_foo` we have, like say `foo_is_thread_safe`/`foo_is_ractor_safe`?
A symbol including the C extension name seems best, to avoid any possible confusion when looking it up.

Maybe there are other ways to mark C extensions than defining symbols, that could still be read by the Ruby implementation reliably?

I used the term `C extensions` but of course it would apply to native extensions too (including C++/Rust/...).

cc @ko1



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>