From: Robert Klemme <shortcutter@...>
Date: 2012-11-08T01:58:28+09:00
Subject: Re: streaming stdout with open4 to prevent parent/child buffer lock

--90e6ba10b07d94685904cdeaa02f
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Nov 1, 2012 at 10:06 PM, Matthew Purdy <lists@ruby-forum.com> wrote:

> i have to fork a process and read a large amount of stdout; however, the
> child buffer gets filled up and everything hangs.  how do you read
> stdout as stream?
>
> the only way i know how to do this is the following two ways; but both
> do not work; how can you get the stdout when the method starts so you
> can read stdout as it get written to from the child process?
>
>
You can use a thread:

$ ruby x.rb
1
2
3
4
5
6
7
8
9
10
$ cat -n x.rb
     1
     2
     3  require 'open3'
     4
     5  Open3.popen3("seq", "1", "10") do |s_in, s_out, s_err, th|
     6    # eat stderr
     7    th_err = Thread.new { s = nil; while s_err.read(1024, s); end }
     8
     9    # copy stdout
    10    s_out.each_line {|line| puts line}
    11
    12    # wait until finished
    13    th_err.join
    14  end
    15

Kind regards

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

--90e6ba10b07d94685904cdeaa02f
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<br><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Thu, N=
ov 1, 2012 at 10:06 PM, Matthew Purdy <span dir=3D"ltr">&lt;<a href=3D"mail=
to:lists@ruby-forum.com" target=3D"_blank">lists@ruby-forum.com</a>&gt;</sp=
an> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left:1px solid rgb(204,204,204);padding-left:1ex">i have to fork a process =
and read a large amount of stdout; however, the<br>
child buffer gets filled up and everything hangs. =A0how do you read<br>
stdout as stream?<br>
<br>
the only way i know how to do this is the following two ways; but both<br>
do not work; how can you get the stdout when the method starts so you<br>
can read stdout as it get written to from the child process?<br>
<br></blockquote><div><br>You can use a thread:<br><br>$ ruby x.rb<br>1<br>=
2<br>3<br>4<br>5<br>6<br>7<br>8<br>9<br>10<br>$ cat -n x.rb<br>=A0=A0=A0=A0=
 1<br>=A0=A0=A0=A0 2<br>=A0=A0=A0=A0 3=A0 require &#39;open3&#39;<br>=A0=A0=
=A0=A0 4<br>=A0=A0=A0=A0 5=A0 Open3.popen3(&quot;seq&quot;, &quot;1&quot;, =
&quot;10&quot;) do |s_in, s_out, s_err, th|<br>
=A0=A0=A0=A0 6=A0=A0=A0 # eat stderr<br>=A0=A0=A0=A0 7=A0=A0=A0 th_err =3D =
Thread.new { s =3D nil; while s_err.read(1024, s); end }<br>=A0=A0=A0=A0 8<=
br>=A0=A0=A0=A0 9=A0=A0=A0 # copy stdout<br>=A0=A0=A0 10=A0=A0=A0 s_out.eac=
h_line {|line| puts line}<br>=A0=A0=A0 11<br>=A0=A0=A0 12=A0=A0=A0 # wait u=
ntil finished<br>
=A0=A0=A0 13=A0=A0=A0 th_err.join<br>=A0=A0=A0 14=A0 end<br>=A0=A0=A0 15<br=
><br></div></div>Kind regards<br><br>robert<br clear=3D"all"><br>-- <br>rem=
ember.guy do |as, often| as.you_can - without end<br><a href=3D"http://blog=
.rubybestpractices.com/">http://blog.rubybestpractices.com/</a><br>

</div>

--90e6ba10b07d94685904cdeaa02f--