From: Robert Klemme <shortcutter@...>
Date: 2010-03-18T21:39:14+09:00
Subject: Re: Thread pool issue

2010/3/18 Sriram Varahan <sriram.varahan@gmail.com>:
> Hello,
>
>
>     I was just trying out the thread pool code form Ruby Cookbook.
>
> I created a script to test it out. The complete code is as follows:
> -----------------------------------------------------------------------------
> require 'thread'
>
> class ThreadPool
>  def initialize(max_size)
>    @pool       = []
>    @max_size   = max_size
>    @pool_mutex = Mutex.new
>    @pool_cv    = ConditionVariable.new
>  end
>
>  def dispatch(*args)
>    Thread.new do
>      # Wait for space in the pool.
>      @pool_mutex.synchronize do
>        while @pool.size >= @max_size
>          puts "Pool is full; waiting to run #{args.join(',')}…\n"
>          # Sleep until some other thread calls @pool_cv.signal.
>          @pool_cv.wait(@pool_mutex)
>        end
>      end
>      @pool << Thread.current
>      begin
>        yield(*args)
>      rescue => e
>        exception(self, e, *args)
>      ensure
>        @pool_mutex.synchronize do
>          # Remove the thread from the pool.
>          @pool.delete(Thread.current)
>          # Signal the next waiting thread that there's a space in the
> pool.
>          @pool_cv.signal
>        end
>      end
>    end
>  end
>
>  def shutdown
>    @pool_mutex.synchronize { @pool_cv.wait(@pool_mutex) until
> @pool.empty? }
>  end
>
>  def exception(thread, exception, *original_args)
>    # Subclass this method to handle an exception within a thread.
>    puts "Exception in thread #{thread}: #{exception}"
>  end
> end
>
> @work = []
> def no_work_only_sleep
>  sleep(2)
>  @work << "Finished"
> end
>
> pool = ThreadPool.new(3)
>
> 10.times do |i|
>  pool.dispatch(i) do
>    puts "Job #{i} started"
>    no_work_only_sleep
>  end
> end
>
> pool.shutdown
> puts @work.length
>
> -------------------------------------------------------------------------
> What I am doing here is creating 10 jobs and within a job calling a
> method.
> Inside the method I am inserting a string "Finished" to an array @work.
> So for 10 jobs this array should contain 10 such strings.
>
> The issue I have here is : For a thread pool of size 3 it gives an
> output of 9 where it should have been 10.
>
> For any other thread pool value it gives 10 as required.
>
> Any reason as to why this is happening?

Two things stand out:

1. as far as I can see you are not properly synchronizing access to @work.

2. IMHO the check of the current thread pool size should be done in
the calling thread in order to block it - not in the newly created
background thread.

Issue 2 is a bit tricky to understand but I believe you have created a
situation where background threads are started and waiting for room in
the pool to become available but the shutdown may get in the way so
you might terminate the pool although there are still threads waiting
to start their work.

Kind regards

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/