From: Eric Wong <normalperson@...>
Date: 2009-12-16T16:46:37+09:00
Subject: Re: working with a slow pipe (IO.popen)

bwv549 <jtprince@gmail.com> wrote:
> On Dec 15, 11:32�pm, Eric Wong <normalper...@yhbt.net> wrote:
> > bwv549 <jtpri...@gmail.com> wrote:
> > > I'm doing some work with pymol and having trouble getting all the
> > > pymol output out:
> >
> > > # I open it in quiet/commandline mode with a pipe:
> > > IO.popen("pymol -cq -p", 'w+') do |pipe|
> > > � pipe.puts "load file.pdb, mymodel\n"
> > > � pipe.puts "run my_script.py"
> > > � # this command will generate a whole bunch of output to stdout
> > > � pipe.puts "my_script mymodel"
> > > � sleep(5) �# <--- this is what I have to do in order to get the
> > > output
> > > � pipe.close_write
> > > � output = pipe.read
> > > end
> >
> > > What is annoying is that unless I sleep for 2-5 seconds I don't get
> > > output or (even worse) my output is cut off. �There has to be a way
> > > besides arbitrarily sleeping to ensure that the command is finished.
> > > I've tried things like piping in "quit", but that doesn't seem to
> > > work. �Anybody know how to solve this? �I'm happy to use a different
> > > approach if it means I can be sure I get all of my output out. � --
> > > Thanks!
> >
> > Hi,
> >
> > I don't know about pymol, but some apps do not respond well to having
> > its stdin closed on it with close_write. �pipe.read should just block
> > until you get output, otherwise put an IO.select([pipe) in front of it.
> >
> > Does this work?
> >
> > � IO.popen("pymol -cq -p", 'w+') do |pipe|
> > � � pipe.puts "load file.pdb, mymodel\n"
> > � � pipe.puts "run my_script.py"
> > � � pipe.puts "my_script mymodel"
> >
> > � � # IO.select([pipe]) # you shouldn't need to uncomment this...
> >
> > � � output = pipe.read
> > � end
> 
> both of those hang indefinitely.  It doesn't hang if I add the line:
> 
>    pipe.puts "quit"
> 
> but that doesn't give me all the output either.  So maybe pymol
> behaves differently than many commandline programs?

Even IO.select([pipe]) hangs indefinitely?  Oh, I guess pymol
expects an explicit quit of some sort.... What if you did:

  IO.select([pipe])
  pipe.readpartial(whatever_size_you_are_comfortable_with)

> Here is what I've worked out for the time being.  It opens a thread
> for reading the output and asks if there is any output every 1/2
> second.  Once we go 1/2 second without output we kill the thread.

Here's a version without threads, should work the same:

  my_string = ""
  Open3.popen3("pymol -cq -p") do |si, so, se|
    si.puts "load file.pdb, mymodel\n"
    si.puts "run my_script.py\n"
    si.puts "myscript mymodel\n"

    # await input for 0.5 seconds, will return nil and
    # break the loop if there is nothing to read from so after 0.5s
    while ready = IO.select([so], nil, nil, 0.5)
      # ready.first == so # in this case

      # read until the current pipe buffer is empty
      begin
        my_string << so.read_nonblock(4096)
      rescue Errno::EAGAIN
        break
      end while true
    end
  end

-- 
Eric Wong