From: stephen.hill@... (Steve Hill)
Date: 2001-07-18T18:37:13+09:00
Subject: [ruby-talk:18039] Re: waiting for `backquotes` to finish - a solution

Thanks for this - I have to admit that I cut down the real code quite
severely to make an example - and I didn't test the example, which was
a bit slack of me. I am glad that people saw the problem I was
alluding to though.

I have to admit that I don't like Guy's solution - when I have several
thousand files to process (honestly!) a 10 second delay per file (or
whatever) does make a difference. I know I can try and reduce the
sleep delay, but it just doesn't hit my elegance spot!

Dave, I do like your solution of running the output directly into
ruby. This will certainly work with this example, and I think I will
take it up; but in general it is still unsatisfactory in the case
where I need to recursively apply the script (Yep, following the
advice in PP (a fab book, thanks) I am using metadata files to control
post processing, and sometimes a recurse through several levels (eg
post process bar.txt -> foo.txt, then post process foo.txt...etc).

Anyway. Also, I'm afraid that on the system I am running ruby on
(HP-UX 10.4), whilst the backquotes process may have completely
finished and returned, all the disk writing certainly hasn't (although
the system I work with has most of the disk space mounted via NFS). So
I really do see problems where when I try reading back in the file, it
ends half way through a line.

I have just tried something along the following lines, and it appears
to work fine.


IO.popen("grep FOO foo.txt") do |pipe|
     pipe.each do |line|
       # ... process and put output into out_str
    end 
end

out_file=File.new("bar.txt","w+")
outfile.puts(out_str)
out_file.close

while File.stat("bar.txt").size!=out_str.size
   sleep 0.001
end
  

Anyway,
thank you both for your good advice and time...guess you guys have
acted as "code teddies" to a certain extent ;-)

Best regards

Steve 


Dave Thomas <Dave@PragmaticProgrammer.com> wrote in message news:<T1X47.9683$PF1.550370@e420r-atl2.usenetserver.com>...
> stephen.hill@motorola.com (Steve Hill) writes:
> 
> > Some of the scripting relies on some of the standard unix tools to do
> > its work, called using backquotes. e.g. the following facet which
> > processes the columns of a file previously created with grep.
> 
> ..
> 
> > The problem is that the script may fail at the line marked A because
> > the file "foo.tmp" has either not been written at all, or has not been
> > completely written.
> 
> This shouldn't be a problem, because the command in backquotes will
> run to completion _before_ the rest of your code is executed.
> 
> However, there's a way to bypass this altogether: you can run the
> underlying command and have your Ruby program read the output
> directly, rather than having to write the output to a file and
> subsequently opening the file. Try:
> 
>  
>   IO.popen("grep FOO foo.txt") do |pipe|
>     pipe.each do |line|
>       # ...
>    end
>   end
> 
> A couple of other points. In your original code chunk, you had
> 'flag? = false'. Does that compile in your version of Ruby?
> You also had
> 
>    unless flag? do
>     ...
>    end
> 
> Although this looks reasonable, it isn't what you meant. The 'do'
> introduces a block (as in an iterator block), not the body of a
> compound statement. This will force 'flag?' to be interpreted as a
> method name.
> 
> 
> Dave