From: Brian Candler <B.Candler@...>
Date: 2003-05-11T19:21:11+09:00
Subject: Re: RCR for child execution

Let me try to see if I can give a specific example of the problem that I
think you're trying to solve:

    - function A is written in C++
    - function A calls function B, which is written in Ruby
    - function B writes to stdout and stderr
    - you want all function B's output to be captured in a string,
      without making any modifications to function B itself

Is that an accurate description of the problem?

If so, it's straightforward to solve. Firstly, note that it doesn't matter
which language A and B are written in. I would use the same basic approach
for these three cases:
1. A and B both in Ruby
2. A in C(++), B in Ruby
3. A in C(++), B in C(++)

So let's start with case 1.

    def functionA
      result = functionB(99)
      puts "The output of B was #{result.inspect}"
    end

    def functionB(arg)
      puts "Hello, world! You passed in #{arg}"
      $stderr.puts "Hello on stderr"
      system("ls /nonexistent")     # so that ls outputs on stderr
    end

    function A

This doesn't work as written of course, as we're not building a result
string. The first line of functionB we could fix by pointing $defout to a
StringIO object, but the second two lines write directly to the Unix stderr
filehandle, and there's nothing we can do about it. We can't modify
functionB, and we certainly can't modify "ls"!

Given that we are not allowed to make any modification to functionB, we have
to write a wrapper and run it in a separate process. That's the only way we
can ensure that while functionB runs it has a different stdout and stderr,
which we can read into a string, without affecting our main program's stdout
and stderr.

Here is the complete pure-Ruby solution:

    def functionA
      result = capture_output { functionB(99) }
      puts "The output of B was #{result.inspect}"
    end

    def functionB(arg)
      puts "Hello, world! You passed in #{arg}"
      $stderr.puts "Hello on stderr"
      system("ls /nonexistent")     # so that ls outputs on stderr
    end

    def capture_output
      # We could use IO.popen("-") to do all this for us, except we want
      # both stdout and stderr attached to the same pipe at the child end
      rd, wr = IO.pipe
      pid = fork do
        rd.close
        STDOUT.reopen(wr)
        STDERR.reopen(wr)
        wr.close
        yield
      end
      wr.close
      result = rd.read
      rd.close
      Process.waitpid(pid)
      return result
    end
      
    functionA  #>> The output of B was "Hello, world! <... etc>"

That's it. You can't guarantee the exact order of the output lines when
mixing stdout and stderr in this way, because they tend to be flushed at
different times, but that's probably not a concern. A more general version
of capture_output would accept a string to be passed to the child's stdin,
and capture stdout and stderr to separate strings.

Now, you have two ways to solve case 2:
a. just use the Ruby capture_output that I have written above

    def wrap_functionB(*args)
      capture_output { functionB(*args) }
    end

The C++ program just calls wrap_functionB instead of calling functionB
directly. This can be done in a more general-purpose way:

    def wrap(method,*args)
      capture_output { method.call(*args) }
    end

so in C++ you first create a method object for the thing you want to call,
and then call 'wrap' with the method object and arguments.

b. rewrite "capture_output" in C++. It maps almost line-for-line to Unix API
calls, so I leave it as an exercise for the reader.

The solution for case 3, where C++ function A calls C++ function B, is the
same as 2b.

Are we getting somewhere now? :-)

Now, as to how useful this approach is in embedding: it might be, if you
have some Ruby code which you want to run which generates output that you
want to capture (your example of Test::Unit's console runner). But you
wouldn't want to do this unless absolutely necessary, because:

- there is an efficiency overhead with starting new processes
- functionB cannot modify any state which is shared with functionA, or even
  between subsequent invocations of functionB; nor can it pass back any
  return value (except a limited integer value can be passed back as the
  child exit status). All results would have to be passed back "in-band" on
  the child's stdout.

So, you can't have your cake and eat it. Either you run functionB as a
separate process, in which case you can capture its stdout/stderr
transparently, or you embed it properly (by calling functionB directly), in
which case it shares its stdout/stderr with the main program, because it
*is* part of the main program.

But there's nothing to stop you calling capture_output in certain cases
only where it's needed, and making direct method calls where you have
control over the Ruby code itself. The latter case is what I think of when
you say "embedded Ruby".

Regards,

Brian.