[ruby-core:44220] Re: Questions about thread performance (with benchmark included)

Hi Urabe, thank you for your input, but I think you have 
over-simplified. Read my inline comments.

Em 09-04-2012 04:24, Urabe Shyouhei escreveu:
> #### MRI threads myths and facts #####
>
> Myth1: "Forking processes to run parallel wastes memory"
>
> The fact is, no.  You can waste memories if you wish, but that is also
> true for threading.  With a  careful implementation a process is quite
> small in  memory consumption.   If you are  in doubt, see  how unicorn
> forks its child processes.

Yes, you're right, there are better ways of using multiple processes 
(through forking) without wasting too much memory like the common 
proxied solution.

But then you have another problem. Suppose you want to benchmark your 
application against multiple Ruby implementations. You'd have to create 
your own "parallel" methods that would be specific for each Ruby 
implementation. For example, forking in JRuby is terribly slow.

> Myth2: "Threading boosts your application performance"
>
> This is also no.  Multi-threaded programming is _very_ difficult to do
> properly.  And  if you  do it badly,  its performance gets  even worse
> than a  single threaded one.   A multi-threaded application  that does
> scale can also scale much easily by using processes.

This is not a myth. This entire thread was born from a real issue:

http://rosenfeld.herokuapp.com/en/articles/ruby-rails/2012-03-04-how-nokogiri-and-jruby-saved-my-week

Multi-thread is not always difficult and being difficult doesn't mean it 
can't be implemented in a more performant way.

> Myth3: "Threading is the way to go"
>
> This is  quite doubtful.  Multi-threaded programming  is too difficult
> to  develop, almost  impossible to  debug,  and normally  do not  work
> properly.   Multi-threading  is too  easy  to  fail.  Some  restricted
> model, such  as Shared Nothing  architecture, seems to have  much more
> potential.

People have been doing multi-threaded programs for decades and while 
certainly there can be bugs hard to track on threaded applications who 
said that programming was supposed to be easy? Sometimes some solutions 
require a more complicated approach.

I agree that Shared Nothing architectures can be interesting too, but 
choosing some solution architecture should be the programmers decision 
in my opinion.

> Myth4: "We are ignoring Rais"
>
> Definitely no.
>
> Myth5: "You can make MRI lock-free"
>
> Do it yourself if you think so.  Patch welcomed.

I don't think any threaded application can be lock-free, including a 
language interpreter. But having locks (instead of a single global lock) 
doesn't mean you can't use the full power of processors.

Unfortunately I don't have time nor skills for patching CRuby even for 
much easier patches... :( Maybe in the future...

I was just stating that I don't think someone with skills has invested 
much time trying to do so either because they don't think CRuby could 
benefit from parallel processing that much.

That is what I'm trying to get them to reflect about this situation when 
I launched this thread. If they agree that parallel threading can be 
very useful, than it would be clear that the only reason for CRuby not 
supporting it is the fact that it is currently difficult to patch Ruby 
to get rid of the global lock approach.

Cheers,
Rodrigo.