From: Eric Wong <normalperson@...>
Date: 2018-01-25T04:34:14+00:00
Subject: [ruby-core:85094] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid

danieldasilvaferreira@gmail.com wrote:
> Hi Eric,
> 
> I've been reading this issue and I'm finding it fascinating.
> Let me play here the role of the ruby developer that is seeking to
> understand better the asynchronous ruby capabilities.
> Every time I read threads(conversations) like this one about the pros
> and cons of Fibers vs Threads I tend to think: stay away from it.
> 
> When people like Kochi write comments like this:
> 
> > "But most (many? some? a few?) of ruby programmer (including me) can not write correct code I believe."
> 
> or Yusuke Endoh:
> 
> > "Thread is considered harmful. Casual Rubyists (including I) had better not use it."
> 
> what these comments make us mere mortals feel?

Often, you will not have to think about things like Threads or
Fibers; and you may use them every day without knowing it.
Fwiw, every project screws up threading (and many other things)
sometimes; even scanning LKML from the past month I see several
subjects with "race condition" in them.

> I will speak about me. When I read such a line I tend to step away.
> So yes, this situation makes me develop single threaded code as much
> as possible.
> I rely on libraries to handle asynchronous behaviour for me and
> specially I rely extensively on the actor model.

Threadlets/Fibers/Threads can all support the actor model.  This
is why I lean towards supporting Queue/SizedQueue but am not as
enthusiastic about increasing scope of Mutex/ConditionVariable.

Threadlet can easily become Actors if matz or ko1 decides to
make such an API.  The implementation details which exist today
would barely change.

I'm not a computer language person; to me it's all just bytes in
memory.  The key difference is "native thread" has support from
an external layer, the kernel, whereas "userspace" Fiber/Threadlet
are invisible to the kernel.

Any actor API can be either "native" or not, or hybrid (M:N threading).
I believe M:N is too unpredictable/controllable to the
programmer (but I could be wrong).

> I doubt I will change my mind unless I start to read that Thread is
> good to be used or Fiber is good to be used.
> 
> When I read all this conversation and you mention corner cases that
> still have problems that is a NO GO for me.

I think the only corner case I mentioned was for libkqueue;
which only affects Linux developers who want to support
some *BSD-specific code without installing FreeBSD.

Normal users won't be expected to use libkqueue.

> IMHO to add yet another Thread like feature it should be "The Killer Feature".

No, what I work towards are incremental improvements and
regression fixes.  So I consider Threadlet a regression fix for
the lightweight Thread we lost in the MRI 1.8 -> YARV (1.9) change.
It is also an opportunity to improve on what 1.8 had with better
scalability and more predictable (safer) behavior.

> The one that we can say to the all community: Hey people use this
> thing because async is a paradise in ruby land at last.

I would never say anything that optimistic :P

> If we don't have this it will be just another Thread, Fiber nightmare
> for the very few who accept the overhead of dealing with all the
> "buts".

Huh?  If you don't like something, you can ignore them and let
others use/try them.  There's plenty of things I don't care for
in Ruby, too.  Sometimes we can deprioritize/deprecate them,
make them less intrusive and move on (see 'callcc', $SAFE, taint).

> And for the record, I use async libraries but I don't feel confident
> about them either knowing that ruby core is not reliable in itself.

I'm not sure what you're talking about.  I suppose nothing is
reliable :P   For example, see how often "stable" Linux kernel
releases come out with GregKH saying "all users must upgrade".
Yet Linux is trusted with countless mission critical systems.

Best we can do is fix bugs and learn lessons from them to avoid
repeating history.

And life goes on...

> Production code in the enterprise world it is not something to mess around.
> For me ruby core needs desperately to change this situation so I
> really hope your work will be the answer for all of this I'm talking
> about.
> So yes, if it is it fits in ruby core like a glove IMO. If it is not
> then we will be much worst because instead of 2 walking deads we will
> have 3.
> A 50% increase is a lot in this domain. Turns things into a joke.

Did you see my other post about blocking accept?  I have every
intent to continue using Thread as-is; and I also use Fiber
as-is in places where it is the perfect tool for the job.

They each have their uses.

And I also look forward to Guilds, too; which I expect to be
implemented using native threads but with less sharing visible
to the Ruby layer.

> So, can you please explain us what peace of mind will we gain with
> this new "light thread" in our everyday work?

But often I want something in-between what Thread and Fiber are,
and that's where Threadlet comes in.

> Thank you very much and keep up the excellent work.
> I appreciate specially the care you have in passing across your
> knowledge on the subject.
> Really helpful and insightful.

You're welcome.

> Note:
> 
> Your last two messages are not part of the issue in redmine. I hope my
> message will be there!

These two?
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/85081
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/85082

Maybe it's a bug in Redmine mailing list integration plugin,
Will try to get with @hsbt to track it down...

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>