From: "Iñaki Baz Castillo" <ibc@...>
Date: 2008-04-18T04:02:14+09:00
Subject: Re: Parallel for loop

El Jueves, 17 de Abril de 2008, Fredrik escribió:
> I added an argument to limit the number of concurrent processes (my
> workstation practically died when I ran all the processes I wanted to
> run):
>
> module Enumerable
>   def forkmap n, &b
>     result = map do |*a|
>       nproc = 0
>       r, w = IO.pipe
>       fork do
>         r.close
>         w.write( Marshal.dump( b.call(*a) ) )
>       end
>       if (nproc+=1) >= n
>         Process.wait ; nproc -= 1
>       end
>       [ w.close, r ].last
>     end
>     Process.waitall
>     result.map{|r| Marshal.load [ r.read, r.close ].first}
>   end
> end
>
> It seems to be doing its job correctly :
>
> irb> Benchmark.realtime { [1,2,3].forkmap(3){|i| sleep(1) ; i * 2} }
> => 1.01134896278381
> irb> Benchmark.realtime { [1,2,3].forkmap(1){|i| sleep(1) ; i * 2} }
> => 3.01262402534485

It's really great. I just see one thing to improve:
The new "n" parameter is mandatory since it's the first parameter. It would be 
nice if it could be not defined (so = infinite):

forkmap(4) { code }  -->  max 4 process
forkmap { code }  -->  max infinite


Do you think your code can be feasible for production enviroments? maybe it 
envolves some danger or risk? If not I suggest you to publish it in any way 
since it's really cool and a missing feature of Ruby. ;)




-- 
Iñaki Baz Castillo