From: "ara.t.howard" <ara.t.howard@...>
Date: 2008-07-02T03:50:52+09:00
Subject: Re: options for running code in parallel


On Jul 1, 2008, at 11:52 AM, Martin DeMello wrote:

> Our testing department wants a small tool that can make a number of
> http calls simultaneously to the same server, the basic logic being
>
> # -----------------------------------------------------------------
> calls = ["call1"..."calln"].map {|x| "http://server/#{x}"}
>
> calls.each do |x|
>  OnSignal(START) do
>    http_send call(x)
>    log_result
>  end
> end
>
> SendSignal(START)
> # -----------------------------------------------------------------
>
> What are the options for doing this cleanly and robustly? I haven't
> been keeping up with the recent work people have been doing on
> distributed/parallel code in ruby, but I get the general impression
> there's been a lot of activity in this area.
>
> martin
>

give this a whack:



cfp:~> cat a.rb
module Threadify
   require 'thread'

   @threads = 8
   @abort_on_exception = true

   class << self
     attr_accessor :threads
     attr_accessor :abort_on_exception
   end
end

module Enumerable
   def threadify opts = {}, &block
     opts = {:threads => opts} if Numeric === opts
     threads = Integer(opts[:threads] || opts['threads'] ||  
Threadify.threads)
     done = Object.new.freeze
     jobs = Queue.new

     each_with_index{|elem, i| jobs.push [elem, i]}
     threads.times{ jobs.push done} # mark the end

     consumers = Array.new threads

     threads.times do |i|
       consumers[i] = Thread.new do
         this = Thread.current
         this.abort_on_exception = Threadify.abort_on_exception
         loop{
           job = jobs.pop
           this.exit if job == done
           args = job.first
           jobs << (job << block.call(*args))
         }
       end
     end

     consumers.map{|t| t.join}
     jobs.push done

     ret = []
     while((job = jobs.pop) != done)
       elem, i, value = job
       ret[i] = value
     end
     ret
   end
end

class Thread
   def Thread.ify enumerable, *args, &block
     enumerable.send :threadify, *args, &block
   end
end


if __FILE__ == $0
   require 'open-uri'
   require 'yaml'

   uris = %w( http://google.com http://yahoo.com http://rubyforge.org/ http://ruby-lang.org 
)

   Thread.ify uris, :threads => 3 do |uri|
     body = open(uri){|pipe| pipe.read}
     y uri => body.size
   end
end



cfp:~> ruby a.rb
---
http://yahoo.com: 9562
---
http://google.com: 6290
---
http://rubyforge.org/: 22352
---
http://ruby-lang.org: 9984




soon to be released as a gem.


a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama