From: stephen.hill@... (Steve Hill)
Date: 2001-09-11T21:19:22+09:00
Subject: [ruby-talk:21082] Re: CGI in Ruby with Apache and Exec

p_hilton@excite.com (Phillip Hilton) wrote in message news:<f50dc966.0108290943.555e7fa9@posting.google.com>...


Phil,
I have added some comments on your fork-exec problem.

> 2: I can't seem to manage to get  'exec' up and running. Here is my
> code where am I going wrong.
> 
> Function: The parent passes 3 variables to multiple child processes.
> Each child is given the same regexpression but a differrent file to
> parse and a different file to write the matches into.
> 
> The parent calls process.wait as many times as there are children.
> (Next part not shown) The parent then scoops up the results in an
> array for output in CGI format.
> _______________________________________________________________________________
> 
> bert=["c:\\westd\\westdb1.txt","c:\\westd\\westdb2.txt","c:\\westd\\westdb4.txt","c:\\westd\\westdb5.txt","c:\\westd\\westdb6.txt","c:\\westd\\westdb7.txt"]
> ernie=["c:\\westd\\resregmatch1.txt","c:\\westd\\resregmatch2.txt","c:\\westd\\resregmatch3.txt","c:\\westd\\resregmatch4.txt","c:\\westd\\resregmatch5.txt","c:\\westd\\resregmatch6.txt,""c:\\westd\\resregmatch7.txt"]
> re_str="((at[ct])|(act))"
> 
> 
> 	(0...bert.length).each  {|i| 
>          exec ("c:\\westd\\cherub3.rb","#{re_str}", "#{bert[i]}",
> "#{ernie[i]}")
> 	 if fork==nil}
> 
>         (0...bert.length).each{|i| i=i-1	 
>          Process.wait
> 	}
> 

OK, your problem here is that you want your main program to stall
untill all the exec'ed processes have finished.

I might add here that I was the idiot who suggested using exec rather
then threading to Phil, because he may be going to a load-balanced set
of machines, where it may be easier to spawn processes onto the
load-balancer and then wait, rather than thread on one machine (OK, I
also know of druby, but have never tried it). I suspect that the
relative merits of threading vs spawning may depend on the exact setup
used in the long run, and where the bottlenecks prove to be.

So what you want to do is to (In Margret Thatcher's words) count them
all out, and count them all back. In this case, you know that you have
spawned bert.length processes, so you can use this as a count, but in
general you might need to count the processes as they go out.

process_count=0
(0...bert.length)arg_array.each do |i|
    if (fork==nil)
        exec ("c:\\westd\\cherub3.rb",re_str, bert[i], ernie[i])
    else
        process_count+=1
    end
end

#now you can wait as the answers come in

while (process_count!=0)
    Process.wait
    process_count-=1
end

Of course, a better (but more complex method - and I appreciate that
your thesis is due in Friday) would be to store a hash of the process
ID's and the data used as you perform the fork calls, so you can
post-process as the exec's return (this is where you might want
multiple threads)

processes=Hash.new
(0...bert.length)arg_array.each do |i|
    pid=fork
    if (pid==nil)
        exec ("c:\\westd\\cherub3.rb",re_str, bert[i], ernie[i])
    else
        processes[pid]=ernie[i]
    end
end

#now you can wait as the answers come in

while (processes.size>0)
    pid=Process.wait2[0]
    
    #now do some post-processing e.g. load the results file
    #and mess about with it
    #may want to do this in a seperate thread
    
    process.delete(pid)
end