From: Charles Oliver Nutter <charles.nutter@...>
Date: 2008-07-12T05:38:51+09:00
Subject: Re: [ANN] threadify-0.0.1

Michael Guterl wrote:
> I am not sure that what I am doing in the code is even reasonable,
> however, I thought it might be worth pointing out.

Ok, there's good news and bad news. First the good news.

I've found several egregious threading bugs in JRuby's Enumerable 
implementation that probably caused the bulk of errors you saw. 
Basically, the runtime information for the main Ruby thread in JRuby was 
getting reused by the blocks passed into threadify, causing all sorts of 
wacky errors (multiple threads all sharing runtime thread data...fun!). 
Fixing that seems to have resolved most of the errors.

Now the bad news...

What you're doing is a bit suspect. In this case, it works out 
reasonable well, since you're just doing a map and gathering results. 
There's some remaining bugs in JRuby wrt the temporary data structure 
used to gather map results (it needs to be made thread-safe) but it can 
work. However in general I don't think this use of threadify is going to 
apply well to Enumera(ble|tor) since so many of the operations depend on 
the result of the previous iteration.

I'll have the remaining issues wrapped up shortly, but I'd love to see 
someone come up with a safe set of Enumerable-like operations that can 
run in parallel. For example, a detect that uses a cross-thread trigger 
to stop all iterations (rather than the naive threadification of detect 
which would not propagate a successful detection out of the thread). 
Things like that could be very useful.

I'd also love to see someone come up with a nice installable gem of 
truly thread-safe wrappers around the core collections, since in general 
I don't believe the core array and friends should suffer the perf 
penalty that comes from always synchronizing.

- Charlie