Re: Point me to help w/ multithreading in 1.9.2-p0

On 28.09.2010 11:16, Alex Stahl wrote:
> No prob... but I'm not sure it's quite what you're looking for.  I'm
> refactoring code I can't get to work instead of finding the root cause
> in the threading.  (BTW - any thread insight would be appreciated based
> on this write-up - cuz there's still problems!  I'm starting to consider
> that a Threaded Load Tester Gem might be handy...)
>
> After staring at the screen for too long, I took a break to ponder
> whether the organization of my threads was fundamentally flawed.  It
> occurred to me that one of the problems I was having - a deadlock -
> could be avoided by instantiating the threads only when needed.
>
> If you review my original code snippet, I spin up all the threads w/ a
> proc object in initialize(), then .stop them, then perform via method
> a .call on each element of the thread's api array, then a .wakeup
> followed by the .join.
>
> ..
> 1.upto @count do
>    @threads<<  Thread.new do
>      Thread.current[:hi5] = Hi5fbapi.new 'redacted params'
>      Thread.current["api_calls"] = []
>      apis.each do |api|
>        Thread.current["api_calls"]<<  get_call(api) #pushes a proc obj
>      end
>      Thread.stop
>    end
> end
> ..
> @threads.each do |thr|
>    thr[:api_calls].each do |api|
>      p api.call thr[:hi5], @session
>    end
> end
> @threads.each {|t| t.wakeup.join}
> ..
>
> On 1.8.7 w/ a single core processor, this is a highly deterministic
> sequence, and would not deadlock.  Once deployed to a multi-core VM
> running 1.9.2-p0 (selected specifically for concurrency), not so much.
>
> There I encountered more deadlocks and also "NoMethodError"s from
> Sinatra (undefined method `bytesize' for #<Thread:0xa36ae0c dead>).
> This would occur during the .each where I would .join.  So it's trying
> to join a 'dead' thread.  Except that if I add anything to prevent that,
> such as
>
> .. unless t.status == 'dead'...
>
> it would still deadlock or NoMethodError.
>
> But further testing showed that adding any operation to the main thread,
> prior to calling .join, would prevent the deadlock:
>
> ..
> @threads.each do |thr|
>    p thr.inspect
> ..
>
> Anyway, I rewrote things where the threads are created in the method,
> not initialize(), so '@api_calls' is already populated by procs, and it
> works fine at small scale:
>
> ..
> thr = []
> 1.upto @count do
>    thr<<  Thread.new do
>      @api_calls.each do |api|
>        p api.call @hi5, @session
>        Thread.pass
>      end
>    end
> end
> thr.each {|t| t.join}
> ..
>
> This runs fine on 1.8.7/single and 1.9.2/multi at like 5-10 threads.
> But when I ramp up to, say, 5,000 (it is a *load* test!), 1.8.7 is fine
> but 1.9.2 segfaults.
>
> Even 500 threads on 1.9.2 is segfaulting right now (but not 1.8.7).  I
> get the handy output:
>
> [NOTE]
> You may have encountered a bug in the Ruby interpreter or extension
> libraries.
> Bug reports are welcome.
> For details: http://www.ruby-lang.org/bugreport.html
>
> I'll write it up tomorrow.
>
> Some open questions:
> 1. When .join is called on a multi-core system, what qualifies as the
> calling thread?  The .main on the main processor, or the thread which
> instantiates my ThreadedLoadTester object?  (i.e. what if
> ThreadedLoadTester is created from a sinatra thread which itself isn't
> main?)
>
> 2. Sinatra regularly reports a 'NoMethodError' for 'bytesize' when the
> last thread is dead but joined to the main thread.  But only when the
> main thread originates w/in sinatra, and not an inline call.
>
> 3. Is there a theoretical maximum to the number of concurrent threads
> which can be created which all access a network interface?  This is
> admittedly a poor theory - what might really cause a segfault in 1.9.2
> when 500 threads all try to access the network?

Without going into too much detail I believe one flaw of your design 
here is that you are not using thread synchronization but instead try to 
explicitly start and stop threads and yield execution.  It may be that 
this is causing your cores, but I really don't know.

What I would do:

1. Use a condition variable to let all threads start at the same time.

2. use Thread#value to collect results.

require 'thread'

lock = Mutex.new
cond = ConditionVariable.new
start = false

threads = (1..10).map do
   Thread.new do
     lock.synchronize do
       until start
         cond.wait(lock)
       end
     end

     # work
     # return results
     [rand(10), rand(100)]
   end
end

lock.synchronize do
   start = true
   cond.signal
end

threads.each do |th|
   p th.value
end

You can probably get away without the condition variable by just 
acquiring the lock (lock.synchronize) in the main thread before you 
create all threads and let all threads just synchronize with an empty block.

Kind regards

	robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/