From: "ara.t.howard" <ara.t.howard@...>
Date: 2008-01-01T03:33:02+09:00
Subject: Re: deadlock in ThreadPool using backtick


On Aug 8, 2007, at 11:11 AM, Justin Johnson wrote:

> Does anyone know what the problem is?

my mind is much too feeble to write robust threaded code that's safe  
on all platforms.  invert your process flow and it's easy: this works  
on *nix and winblows ->


cfp:~ > cat a.rb

require 'thread'

class ThreadPool < ::Array
   Done = Object.new.freeze

   attr 'q'
   attr 'exception_handler'

   def initialize size = 42, &exception_handler
     @q = Queue.new
     @exception_handler = exception_handler || lambda{|e| raise}
     Integer(size).times{ push new_consumer }
   end

   def new_consumer
     Thread.new do
       Thread.current.abort_on_exception
       loop do
         object = q.pop
         break if object == Done
         begin
           object.call
         rescue Exception => e
           exception_handler.call e
         end
       end
     end
   end

   def dispatch &block
     @q.push block
   end

   def shutdown
     size.times{ q.push Done }
     each{|t| t.join}
   end
end


null = test(?e, "/dev/null") ? "/dev/null" : "NUL"
pool = ThreadPool.new 2

begin
   1.upto(200) do |i|
     pool.dispatch do
       puts "Job #{ i } started."
       command = "ruby -e' fill_stdout_pipe = 42.chr * 8193; puts  
fill_stdout_pipe '"
       ` #{ command } `
       system "#{ command } > #{ null } 2>&1"
       puts "Job #{ i } stopped."
     end
   end
ensure
   pool.shutdown
end




by spooling up all the thread at once you'll save the overhead of  
creating all those threads.  the @q.pop stuff does all the condition  
variable stuff you were trying to do so you can check that code out  
if you want to.

ps - when testing stuff that reads output (backtick, popen, or  
blatant io) be sure to test will a full pipe on stdout - typically >  
8192 bytes.


kind regards.

a @ http://codeforpeople.com/
--
it is not enough to be compassionate.  you must act.
h.h. the 14th dalai lama