[ruby-core:86853] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid

[e-mail] wrote:
> Eric Wong wrote: 
> > yahns is designed to deal with apps with both slow and fast
> > endpoints simultaneously. Given N threads running, (N-1) may be
> > stuck servicing slow endpoints, while the Nth one remains free
> > to service ANY other client.

> I'm surprised by this, how would slow clients be a problem?

I did not mention "slow clients", I said "slow endpoints".
Some endpoints make expensive DB queries and so take longer.
Endpoint speed and client speed are totally different.

> You can work around TTY problems using something like
> http://code.saghul.net/index.php/2016/05/24/libuv-internals-the-osx-select2-trick/

Yes, that's what I had in mind (along with various char devices,
fifos and all the other oddball corner cases).  Maybe an extra
thread won't be necessary, even, if one can select/poll directly
on the kqueue FD.

> > Really, I think it's a waste of time and resources to support
> > these things.
> 
> If you think this, why add this proposed feature to Ruby at all.

It's to fix the lack of lightweight concurrency we lost with 1.9
I wish I didn't have to, but I'd rather keep using Ruby and
not depend on a truckload of out-of-tree dependencies to fix
a regression.  Leftpad, anyone?

> Thread.new do
>     Thread.current.selector = NIO::Selector.new
>     # selector responds to :wait_for_single_fd
> 
>     # Thread at exit calls selector.run

I don't expect a user to have to care or know what a "selector"
is.  I try to avoid tuning knobs because they cause cognitive
overhead for users (and they become out-of-date and wrong).  I'd
rather try to make things work well out-of-the-box.

I think having a choice of selectors will make the problem
worse because we'd have more variations to deal with when
users report bugs.

> As has been demonstrated, there are lots of trade-offs.
> Personally, I'd rather use libev or libuv which is actively
> maintained than what you've proposed here. They cover a much
> larger chunk of functionality, and they are maintained and
> updated independently of Ruby. Why repeat all that work? Are
> you going to maintain this feature for the next 20 years?

Many reasons:

- core features of Ruby shouldn't depend too much on 3rd party code.
  I recall matz has a similar sentiment (so no LLVM, either)

- I consider lightweight concurrency a core feature of a language.
  Languages like Go and Erlang have won over Ruby users with
  good lightweight concurrency out-of-the-box.

- libev and libevent consider multithreading an afterthought
  (not sure about libuv, here)

- GVL or not, Ruby needs to continue to work well with many
  native threads, so having a 1:1 nativethread:reactor relationship
  is unreasonably expensive.

- libev ev(3) manpage has a misinformed rant on epoll.  That tells
  me more about how bad libev is at using epoll than epoll itself.

- contributing to libuv requires GitHub, a proprietary communications
  platform that's off limits to me

- the rest of the Ruby core team can help maintain this.
  I don't expect to live 20 years, but I do my best to
  document the internals while I'm around.

- some of us can fix bugs and improve the Linux kernel for this.
  Maybe we'll get some *BSD kernel hackers on board, too.

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>