From: Michal Suchanek <hramrach@...>
Date: 2011-05-14T00:32:22+09:00
Subject: Re: Shell pipeline in Ruby?

On 12 May 2011 18:05, Michal Suchanek <hramrach@centrum.cz> wrote:
> Hello,
>
> how do you write an equivalent of
>
> $ cmdA | cmdB | cmdC
>
> in Ruby?
>
> Specifically, I would like to see the PID, return value and stderr of
> each of these commands but I would like cmdB to read the stdout of
> cmdA directly, on its stdin, and similarily for cmdC and cmdB.
>
> Here I am not interested in feeding cmdA some particular input (eg. it
> can read /dev/null for all I care) but in general gluing arbitrary fd
> to its stdin might be desirable in other cases.
>
> Thanks
>
> Michal
>

OK, thanks for all the replies.

It seems this is not doable with ruby 1.8 but spawn and similar
methods in 1.9 grew the named arguments which include the file
descriptors.

Here is a working (although not polished) solution for ruby 1.9.

First a short testing script called cmd:

#!/usr/bin/ruby

prefix = ARGV[0] + ":"
append = ARGV[1]

while l = STDIN.gets do
  STDOUT << l
  STDERR << prefix << l
end

STDOUT.puts append

And the long script with pipes that calls the above:

#!/usr/bin/ruby1.9.1
# Debian calls ruby 1.9 ruby1.9.1

pipes = []
(1..4).each {pipes << IO.pipe}
errs = []
(1..3).each {errs << IO.pipe}

writer =  pipes[0][1]
reader =  pipes[3][0]
i = 1

# Careful about pipe direction, it only works one way.
pids = [
spawn("cmd", "a", "aa", :in => pipes[0][0], :out => pipes[1][1], :err
=> errs[0][1] ),
spawn("cmd", "b", "bb", :in => pipes[1][0], :out => pipes[2][1], :err
=> errs[1][1] ),
spawn("cmd", "c", "cc", :in => pipes[2][0], :out => pipes[3][1], :err
=> errs[2][1] ),
]

# We don't really want to see these pipes, perhaps some
Open3.pipeline_* method would create them out of sight
pipes.flatten.select{|p| p != writer && p != reader}.each{|p|p.close}\

maxlen = 128 # some arbitrary size for chunk of data read at once

outputs = errs.collect{|r,w|r}
outputs << reader
wsel = [writer]

while true do
  ios = IO.select(outputs, wsel, nil, 1)
  STDERR << '.'

    string = "%4i\n" % i
    if writer then
      begin
        result = writer.write_nonblock(string)
        i = i + 1
        if ( i > 1000 ) then
          writer.close
          writer = nil
          wsel = nil
        end
      rescue IO::WaitWritable, Errno::EINTR
      end
    end

  outputs.each{|io|
    begin
      result = io.read_nonblock(maxlen)
      STDOUT << result
    rescue IO::WaitReadable, Errno::EINTR, EOFError
      if $!.is_a? EOFError then
        STDERR.puts "EOF"
        outputs.reject!{|o| o == io }
      end
    end
  }

  (0...3).each{|i| p = pids[i]
    if p and Process.waitpid(p, Process::WNOHANG) then
      STDOUT.puts $?.inspect
      pids[i] = nil
    end
  }
  break if !ios && pids.select{|pid| pid} == []
end