From: Fredrik <fredjoha@...>
Date: 2008-04-17T12:15:06+09:00
Subject: Re: Parallel for loop

I added an argument to limit the number of concurrent processes (my
workstation practically died when I ran all the processes I wanted to
run):

module Enumerable
  def forkmap n, &b
    result = map do |*a|
      nproc = 0
      r, w = IO.pipe
      fork do
        r.close
        w.write( Marshal.dump( b.call(*a) ) )
      end
      if (nproc+=1) >= n
        Process.wait ; nproc -= 1
      end
      [ w.close, r ].last
    end
    Process.waitall
    result.map{|r| Marshal.load [ r.read, r.close ].first}
  end
end

It seems to be doing its job correctly :

irb> Benchmark.realtime { [1,2,3].forkmap(3){|i| sleep(1) ; i * 2} }
=> 1.01134896278381
irb> Benchmark.realtime { [1,2,3].forkmap(1){|i| sleep(1) ; i * 2} }
=> 3.01262402534485

/Fredrik