From: Michal Suchanek <hramrach@...>
Date: 2008-08-15T01:35:20+09:00
Subject: Re: Threaded IO trouble

So it turns out the application I am using has some provisions for
running in terminal. If I run just the application from a shell it
behaves much like cat - it processes lines as I type them except it
buffers one line. If I wrap it in iconvs or cats it buffers much more,
and the only way to get output is to close the input pipe.

So I would really like to get ruby to use pipes properly.

I can reproduce deadlocks even when I replace the application with cat:

$ cat testt.rb
def try_analyze *words
  analyzer = IO.popen( 'cat', IO::RDWR | IO::SYNC )
  res = []

  t = Thread.new( (IO::for_fd analyzer.fileno) ,res){|fd,ary|
    while l = fd.gets do
      STDERR.putc 8
      ary.push l
    end
    fd.close rescue nil # hopefully prevents zombie hordes
  }

  words.each{|w|
    STDERR.putc '.'[0]
    analyzer.puts w
  }

  analyzer.close_write rescue nil
  t.join
end

1.upto(10000){|_|
  try_analyze *%w(a b c d e f g h i)
}

$ ruby -w testt.rb
testt.rb:23: warning: `*' interpreted as argument prefix
....cat: stdin: Bad file descriptor
.deadlock 0x5154: sleep:F(4)  - testt.rb:6
deadlock 0x318f8: sleep:S (main) - testt.rb:15
testt.rb:15:in `write': Thread(0x318f8): deadlock (fatal)
        from testt.rb:15:in `puts'
        from testt.rb:15:in `try_analyze'
        from testt.rb:13:in `each'
        from testt.rb:13:in `try_analyze'
        from testt.rb:23
        from testt.rb:22:in `upto'
        from testt.rb:22

The other way around the code looks much simpler but it really locks up:

$ cat testt2.rb
def try_analyze *words
  res=[]
  analyzer = IO.popen 'cat', IO::RDWR + IO::SYNC

  t = Thread.new(analyzer, words){|fd,ary|
    ary.each{|w|
      STDERR.putc '.'[0]
      fd.puts w
    }
    fd.close_write
  }

  while l = analyzer.gets do
    STDERR.putc 8
      res.push l
  end
  fd.close rescue nil # hopefully prevents zombie hordes
  res
end

1.upto(10000){|_|
  try_analyze *%w(a b c d e f g h i)
}

$ ruby -w testt2.rb
testt2.rb:22: warning: `*' interpreted as argument prefix
^Ctestt2.rb:3:in `popen': Interrupt     from testt2.rb:3:in `try_analyze'
        from testt2.rb:22
        from testt2.rb:21:in `upto'
        from testt2.rb:21


Any suggestions how to make pipes work in ruby are welcome.

Thanks

Michal