From: Eric Wong Date: 2018-08-14T17:47:02+00:00 Subject: [ruby-core:88484] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid Koichi Sasada wrote: > On 2018/08/14 9:42, Eric Wong wrote: > > I am thinking of adding preemption support to this feature for > > compatibility with 1.8 > > So that "auto-fiber" proposal is to provide green threads like Ruby 1.8? Yes; this was always the idea. > Like: > ``` > model 1: Userlevel Thread > Same as traditional ruby thread. > ``` > in thread.c comment (I wrote 13 years ago!). > > I don't against this idea, but I think it is hard to select these options by > Ruby programmers. I think changing Thread implementation model from native > thread (1:1 model) to green thread mode (1:N model) is better for Ruby > programmers. No, I want to keep current Thread 1:1 model because it is useful for filesystem operations and some CPU/memory intensive tasks (zlib). Changing "Thread" now to 1:N would also break code written for 1.9..2.5 1:N model is good for most network operations (high latency, low throughput). > To change them, we need to discuss pros. and cons. of them carefully. There > are several good points (the biggest advantage of 1:1 model is friendly for > outer libraries) but are bad points (1:1 model has performance penalties, > and recent glibc malloc arena issues and so on). Agreed. > I don't think it is a good idea to choose such internal implementation by > Ruby programmers. ... easy? I think it is necessary to give that control to programmers. We can educate them on pros and cons of each and to use them in combination. ``` 1:1 + C-ext parallelism for SMP systems (example: zlib) + filesystem parallelism (no non-blocking I/O on FS ops) + external library compatibility + compatibility with 1.9..2.5 code - high memory use (malloc arenas, kernel structs) - kernel resource limitations (Process::RLIMIT_NPROC) 1:N + C10K problem (or C100K :P) + compatibility with old 1.8 designs + low memory and resource use (malloc and kernel never sees extra data structures) - cannot take advantage of SMP or multiple filesystems alone ``` The key is a programmer may combine 1:1 and 1:N for different parts of the program flow. So where the program is bottlenecked on filesystem, it can rely on 1:1 behavior, but when the program is waiting on slow network traffic, it can rely on 1:N behavior For example: I have 4 filesystems, I might have 32 native threads (8 threads/FS to keep kernel IO scheduler busy). But I will still serve 100K network clients with 100K 1:N threads and that can even use 1 native thread. Unsubscribe: