From: Phillip Gawlowski <pg@...>
Date: 2010-01-08T05:37:45+09:00
Subject: Re: Non-blocking communication between Ruby processes

On 07.01.2010 20:50, Robert Klemme wrote:
> On 01/07/2010 08:01 PM, Phillip Gawlowski wrote:

>
> That's something different than you proposed initially, isn't it? This
> approach (increasing the number of readers if the pipe fills too fast)
> is better because it regulates read performance according to load.

A little refined (in that I skipped the buffering), but it's still the 
same core: check the pipe, and sin off new threads as needed.

> IMHO this approach (local buffering if the pipe cannot be written to) is
> not really helping because the pipe *is* a buffer already. In other
> words, the same effect will happen - only later. The only argument in
> favor of additional buffering I can see is less lock contention: if
> every writer process has multiple threads that want to write to the
> buffer, they could instead write to a Queue internally and a single
> reader could read from that local queue and write to the global queue.
> That would reduce the number of writers that compete for locks on the
> global queue. Whether that is performant or not would need to be tested.

This might be a difference in interpretation: I see the pipe in this 
instance as a simple inter-process communication solution, not per se a 
buffer.

Otherwise: You are right.

Also in that performance would've to be tested, and the constraints have 
to be known (I�aki already mentioned that getting all data is less 
important to him, so buffering wouldn't be strictly necessary, either).

> Nevertheless I would start with a simple solution, monitor its
> performance and change the implementation if it does not scale well
> enough. Often simple solutions work surprisingly well... :-)

Indeed. And it's easier to iterate from something simple, than to 
iterate from something complex, too. ;)

--
Phillip Gawlowski