From: Peter Hickman <peterhickman386@...>
Date: 2012-09-03T22:25:15+09:00
Subject: Re: working on multiple machines in a LAN

On 3 September 2012 12:59, ajay paswan <lists@ruby-forum.com> wrote:
> Just  like a stupid question, how can I write to 'job_1234_worker_1.txt'
> on runtime, which will be accessed by a worker too?

We we are assuming a common directory where both the boss and worker
can read and write to, you said that this was on a lan. The the boss
would do something like

tmp_filename = "#{COMMON_DIRECTORY}/job_#{job_number}_worker_#{worker_id}.tmp"
real_filename = "#{COMMON_DIRECTORY}/job_#{job_number}_worker_#{worker_id}.txt"
f = File.new(tmp_filename, "w")
f.puts ... # Write the contents of the file
f.close

File.rename?(tmp_filename, real_filename)

This should stop the worker being able to read the file before the
boss has finished writing to it. Then the worker just has to

while true
  Dir["#{COMMON_DIRECTORY/job_*_worker_#{my_id}.txt"].each do |file|
    # Read the contents of the file and extract the job_id from either the
    # filename or the contents of the file itself.
    # Do the work
    # Write the results to
"#{COMMON_DIRECTORY}/results_#{job_id}_worker_#{my_id}.txt"
    # in a similar manner as above 'write to .tmp, rename to .txt'
    File.delete(file)
  end

  sleep 60 # We have processed all the available jobs, lets wait a
minute before we look again
end

Basically the worker just monitors the common directory, picks up work
and writes results. The boss however has two tasks to undertake. 1)
make sure that there is enough work in the common directory for all
the workers so that none are idle 2) Read and process the results.

The worker is very simple, the boss can get quite complex if you need
more than one worker to perform the same job to crosscheck the results
but even then it is not a biggie.