From: Robert Klemme <shortcutter@...>
Date: 2010-01-05T20:45:08+09:00
Subject: Re: "Dummy" IO object to push and pull data?

On 04.01.2010 14:38, Shay Hawkins wrote:
> Brian Candler wrote:
>> Sorry, I've read that several times and I still don't understand. Can 
>> you post a small example which demonstrates the problem, preferably 
>> standalone?
> 
> My apologies: I'll see if I can elaborate a bit...
> 
> Well, I originally was trying to find a workaround to this, but it seems 
> the root of the problem is in the process' streams. I have a seperate 
> program that I did not create myself (and therefore cannot change the 
> code of) - this program puts output into stderr, and accepts input 
> through stdin.
> 
> I call Open3.popen3 to create the three streams. In a loop, I want to 
> constantly read input from stderr. (Assuming I assigned the pipes to 
> stdin, stdout, and stderr variable names.)
> 
> loop do
>    line = stderr.gets()
>    puts("Output: #{line}")
> end
> 
> Then, if I receive an output of say, 0, I want to input something to the 
> program through stdin five seconds later.
> 
> loop do
>    line = stderr.gets().strip()
>    puts("Output: #{line}")
> 
>    if(line == "0")
>       Thread.new() do
>          sleep(5)
>          stdin.puts("Received 0 five seconds ago.")
>       end
>    end
> end
> 
> It creates the thread properly, but when the sleep expires and it is 
> ready to call stdin.puts(), it cannot do anything, because the loop 
> continued around and it is reading from stderr.

That can't normally be, because the thread runs independently of the
loop.  I am rather suspecting that you are having an issue with not
waiting for all threads to proper finish or do not make sure pipes are
read properly.  You could also have created a dead lock that way, i.e.
by not reading all pipes properly because a process that tries to write
to a filled pipe is blocked.

I have created a pair of test scripts which I will place below. 
Strangely enough, there seems to be an issue with detecting closing of 
the stderr pipe which I need to research separately.  If you just invoke 
the client script with "./clnt.rb 15 x" you will see the interaction 
properly working.  If you, for example, increase the number and omit the 
reading of sin in the client the server will eventually block in one of 
the puts or printf statements.  It may be that this is what you are 
observing.

Kind regards

	robert


Place both files in the same directory.

file clnt.rb:
#!/usr/local/bin/ruby19

# "client" which starts the server and every time a
# zero is read from the server's stderr pipe we send
# a notification to the server's stdin.
# server's stdout is used for logging state and is mirrored
# to this process's stdout with prefix "From server".

require 'open3'

# for debugging
Thread.abort_on_exception = true

Open3.popen3 File.join(File.dirname($0), "serv.rb"), *ARGV do |sin, 
sout, serr|
   sin.sync = true

   # we must make sure all pipes are either read or closed!
   r = Thread.new do
     sout.each do |line|
       puts "From server: #{line}"
     end
   end

   threads = []

   serr.each do |line|
     printf "%-20s read %p\n", Time.now, line

     line.chomp!

     if line == "0"
       threads << Thread.new do
         sleep 5
         sin.puts "Received 0 five seconds ago."
       end
     elsif /finish/i =~ line
       # interestingly enough EOF detection does not
       # work with serv.rb
       break
     end
   end

   puts "finished reading"

   threads.each {|th| th.join}

   puts "finished notifying"

   # init shutdown sequence
   sin.close
   r.join
end


file serv.rb:
#!/usr/local/bin/ruby19

# "server" process which writes a fixed amount of numbers
# to stderr and then closes stderr to indicate it's finished.
# first arg is no of repetitions
# second arg if present says indicate end with a particular
# message to stderr.

$stdout.sync = $stderr.sync = true

puts "started"

th = Thread.new do
   $stdin.each do |line|
     printf "%-30s read %p\n", Time.now, line
   end
end

rep = (ARGV.shift || 10).to_i
ind = ARGV.shift

rep.times do
   $stderr.puts rand(3)
   sleep 1
end

puts "finished writing"

# indicate we are done
$stderr.puts "Finish" if ind
$stderr.close
puts "stderr.closed? #{$stderr.closed?}"

puts "finishing"

th.join

puts "finished"



-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/