Re: Ruby and science ?

On Sat, Dec 18, 2010 at 9:55 PM, James Edward Gray II
<[e-mail]> wrote:
> Of course, like anything, there are tradeoffs and JRuby sucks at other th=
ings, like manipulating processes in a POSIX environment. =C2=A0I don't use=
 it in these scenarios and you know that I've filed bugs for the specific p=
roblems I've run into (some of those have been partially addressed).

The JVM and the JDK APIs suck at process manipulation...not JRuby.
JRuby does the best job it can do cross-platform with the JDK APIs
provided for it. If you need to go outside those APIs, or if we "suck"
in how we utilize them, it's a trivial matter to bind native C
process-management logic via FFI and use that. It won't be as portable
as what we provide, but it will work.

Providing the excellent cross-platform experience the JVM provides
(and which JRuby provides by extension) means a lot of
platform-specific things are a bit cumbersome. Our direction has been
to provide the cross-platform experience and allow people to opt out
of portability through FFI if necessary. You may disagree with that
approach.

> All that said, I think you were pretty harsh on using processes for concu=
rrency in general. =C2=A0That "blunt tool" is pretty much the core of the U=
nix operating system, which I think a lot of us are found of. =C2=A0I often=
 find it easier to work with processes that threads myself, though obviousl=
y some programmers think the other way.

Processes for concurrency works great. The blunt tool I meant was how
you get those processes to coordinate. You basically have a handful of
cumbersome options:

* Signals, which can't communicate much data
* Streams, pipes, files, shared memory, which can only carry byte[]
data, requiring marshaling

With threads, it's possible to communicate between concurrent
processes using normal OO constructs like queues, actors, and simple
method calls. You can emulate that with processes using one of the
above mechanisms, but it's a leaky abstraction. On the other hand,
your queues, actors, and method calls across threads need to be
thread-safe. Tradeoffs.

JRuby is perfectly happy to work with a multi-process model, but you
may need to opt out of portability to get the lowest-level behaviors
of a typical UNIX environment. I personally have nothing against
processes. Threads are just easier, if you stay out of the danger
zones.

> On the contrary, threading is so challenging to get right that "threading=
 is hard" is a popular saying:
>
> =C2=A0http://www.google.com/search?q=3D%22threading+is+hard%22

Threading is hard if you do it wrong. The problem is that it's easy to
do it wrong.

Follow these rules and threading is a very nice, very clean, very easy
way to do concurrency:

1. Don't share data
2. If you must share data, don't share immutable data
3. If you must share mutable data, guarantee ACID (atomicity,
consistency, isolation, durability)

Clojure is a perfect example of an environment that uses threads
heavily by defaulting to (2) and providing software transactional
memory for (3). Other than enforcing immutability, nothing Clojure
does for concurrency could not be done in Ruby. Anyone interested in
seeing concurrency done the Clojure way with JRuby can find many
examples online.

Threads "fail" in that none of these rules are enforced at any level.
They're a very sharp tool with many dangerous paths. But I prefer
sharp tools.

> It bugs me that people are so harsh on fork(). =C2=A0I avoided it like th=
e plague when I was a younger programmer because everyone had me convinced =
it was evil. =C2=A0I'm now far more dangerous because I took the time to le=
arn it and understand it. =C2=A0I strongly recommend all programmers do the=
 same. =C2=A0(By the way, ara.t.howard taught me most of what I know about =
processes, directly and indirectly!)

I have no problem with fork. If JRuby could support fork on the JVM,
we would do so. We don't only because all mainstream JVMs spin up
multiple threads, which are not carried along to forked child
processes (and even if they could be restarted, it's a very
complicated transition that might defeat much of the benefit of
forking).

> So JRuby is good at threads and not so good at processes, in my opinion. =
=C2=A0Processes are also not at all evil. =C2=A0Judge not lest ye be judged=
. =C2=A0;)

It might be more correct to say that the JVM is good at threads and
not so good at processes, nothing that JRuby makes it possible via FFI
to be nearly as good at processes as any POSIX application. We have
simply prioritized making JRuby work uniformly across platforms first,
while still providing the tools people need to opt out of portability
for lower-level behaviors and features.

- Charlie