Re: Concurrent Ruby?

On Monday 28 July 2008 23:17:22 Kyle Murphy wrote:
> With Matz supposedly making Ruby 2.0 right now, is it possible to make
> it concurrent like Erlang

Not like Erlang, no.

Erlang does a couple of things differently. The most obvious one, which makes 
it so scalable, is the message-passing -- Erlang uses "processes" and 
message-passing almost as a programming paradigm. We talk 
about "Object-Oriented Programming"; Erlang people talk 
about "Concurrency-Oriented Programming".

These are much easier to write and scale than threads, and they perform much 
better than single threads.

There are a few of us working to rectify this situation, at least 
semantically -- there's Revactor, Dramatis, and my own unreleased project 
which I've been wasting a few weekend hours on.

Another reason, which I'm running into while working on the above project, is 
that Erlang has no mutable data. It even goes so far as to make variables 
single-assignment, which is just annoying, but the data structures themselves 
are never changed. Take a simple (contrived) Ruby example:


def some_function(options={})
  options[:foo] ||= 'Foo'
  options[:bar] ||= 'Bar'
  options[:foobar] ||= options[:foo] + options[:bar]

  some_file.each_line do |line|
    line.chomp!
    line.gsub! /curses/i, '******'
    puts line
  end
end


See, we're changing things. Arrays, strings, whatever -- it's actually the 
characters inside the string that are changing.

In Erlang, (almost) no data ever changes, you just create new data. Which 
means that when you send a message to another process, it's as simple as 
sending a pointer across -- which means it's not only a constant-time 
operation, it's an absurdly cheap constant-time operation. So the data is 
shared, but because it never changes, you don't have to lock it.

Which means that in Erlang, message-passing is so cheap we don't have to worry 
about it. If we ported the message-passing to Ruby, it's either unreliable or 
it's massively expensive and still somewhat unreliable. I'm not sure there's 
a good way around this, though if there is, I intend to find it.

> so as to take advantage of the future 
> multi-core devices?  Thank you.

This might happen -- maybe, sort of. Keeping all of the above in mind, 
threading in Ruby is modeled after the traditional C and Java model, which 
means they're probably more expensive to create, and certainly more 
dangerous, which means there won't be as many of them.

On top of all that...

Right now, Ruby shares a problem with Python called the GIL -- the Global (or 
Giant) Interpreter Lock. What this means is that only one Ruby instruction 
may execute at a time. So even though they're using separate OS threads, and 
even though different Ruby threads might run on different cores, the speed of 
your program (at least the Ruby part) is limited to the speed of a single 
core.

The standard response, which you'll probably already see (since I'm taking the 
time to write a longer answer), is that you can do threading in two ways: 
Either fork off a whole new Ruby process, so you probably can't have any 
shared-memory problems -- and/or write the expensive parts in C, and have 
your C extension release the Ruby GIL.

(See, you can have more than one bit of C code running in a Ruby program at 
once, even alongside all the Ruby stuff -- at least until they need to do 
something with Ruby itself.)

There's also JRuby, which uses Java's native threads, and has no GIL. There 
have been some problems with them lately, but they should work -- but again, 
keep all of the above in mind. You'll be threading as well as Java does, not 
as well as Erlang does.

As you can probably tell, I'm not really happy about all of this.

Now, unlike Python, it looks as though the Ruby GIL might eventually be 
removed. And there is JRuby. And there's the various actor projects (mine 
included). So it's conceivable that we'd get Ruby scalable to arbitrary 
numbers of processors.

But again, I suspect Erlang is still going to do it better, if all you care 
about is multicore and efficiency. (Ruby is doing a better job of Unicode, 
has much more library support, and I much prefer its syntax.)