From: Robert Klemme <shortcutter@...>
Date: 2009-03-17T05:22:36+09:00
Subject: Re: does IO.read block?

On 16.03.2009 21:00, Michael Malone wrote:
>> One problem that could hit you is that you might not get the 
>> concurrency right: you need to make sure that stderr and stdout are 
>> read concurrently (if there can be output on both) because if the pipe 
>> on one of them fills up your child process is blocked.  Maybe your 
>> issue lies in that area.
>>
> I have set STDOUT.sync = true  Is that enough?

No, definitively not!  This covers the *sending* side but the locking 
issue I described is caused by the receiving side not reading all the 
data.  Assume an application as a child process that frequently writes 
to stdout and stderr.  The parent process at the other end of those two 
(!) pipes only reads the stdout pipe.  Now what happens is this: once 
the pipe's buffer (OS and configuration dependent, often 4k) fills up 
the write call blocks and the child hangs - indefinitely.

>  It should be, because I 
> don't have any output originating from the child processes or any child 
> threads, only the parent/calling thread.  But I've been wrong before...
> 
> Though I think I have narrowed the problem _very_ slightly.  The 
> thread/process pair that blocks indefinitely blocks on the 
> read_end.read() call.  Which is weird and annoying.

And you are sure you close stdout in the client or the client terminates?

>  My next task will 
> be to set up some logging to figure out if there should have been an 
> exception to chuck down the pipe. (In my test case I have a random 
> amount of jobs, in which every second one raises an exception)

What I usually do in these situations that can be a bit tricky: I start 
out writing a simple test scenario that I modify step by step until it 
resembles the problem I want to solve _and_ does not exhibit abnormal 
behavior.

> Thanks a lot for your help!  It's certainly possible that this is going 
> to be my "Stupidly Difficult Bug When I Was Near The Beginning Of My 
> First Programming Job" that everyone seems to have.

LOL

	robert