From: ahoward <ahoward@...>
Date: 2003-01-28T01:57:29+09:00
Subject: Re: solaris porting problem -- flock failure?

On Mon, 27 Jan 2003, ts wrote:

> >>>>> "a" == ahoward  <ahoward@fsl.noaa.gov> writes:
>
> a> also, earlier you said that flock was 'broken' on solaris.  what exactly did
> a> you mean by that?  supposedly solaris uses fcntl to implement flock so why
> a> does ruby need it's own?  i'm sure there is an answer i'm just curious...
>
>  Which version of Solaris ?

SunOS 5.6 sun4u
SunOS 5.7 sun4u
SunOS 5.8 sun4u
SunOS 5.9 sun4u
etc...

i think we have them all.  ;-)

>  If I'm remember correctly, on some version flock() work if the file is
>  open for writing.

all the above versions' man pages claim this to be the case - a really weird
requirement since you may only wish to ensure repeatable reads...

>  You need a #rewind after you have got a lock.

huh?  you mean when writing C programs or also from within ruby?

-a

ps.  here is an update to my earlier post regarding a working coordination of
multiple processes/threads that works on linux and solaris.  the earlier post
had one significant bug (though it still worked) :

the line

  while ((file.flock File::LOCK_EX | File::LOCK_NB)) do
    Thread.pass
  end

should, of course, have been

  while (not (file.flock File::LOCK_EX | File::LOCK_NB)) do
    Thread.pass
  end

note the 'not' !!

here is an updated version for anyone whos interested.  again, i'd be
interested to hear if anyone ran it successfully on 'your' platform.

file : feeding_frenzy.rb

----CUT----
  #!/usr/bin/env ruby

  # this program coordinates access to an output file for the purpose of #
  # inserting a timestamp between multiple threads spawned from within multiple
  # forked processes.  the coordination is considered successful if no
  # timestamps are inserted non-sequentially

  require 'thread'
  require 'ftools'

  pid = Process.pid

  n_process  = (ARGV.shift or 4).to_i
  n_thread   = (ARGV.shift or 4).to_i
  n_mark     = (ARGV.shift or 2 << 9).to_i
  outpath    = (ARGV.shift or '/tmp/out')
  sortedpath = (ARGV.shift or '/tmp/sorted')


  File.rm_f outpath
  File.rm_f sortedpath

  n_mark_per_thread = n_mark / n_process / n_thread
  cids              = []

  $stdout.sync = true
  $stdout.puts "================"
  $stdout.puts "N_PROCESS : #{n_process}"
  $stdout.puts "N_THREAD  : #{n_thread}"
  $stdout.puts "N_MARK    : #{n_mark}"
  $stdout.puts "OUTPATH   : #{outpath}"
  $stdout.puts "================"
  $stdout.puts "\n#{pid} STARTED"


  n_process.times {

    cid =
      Process.fork {
        ppid = Process.ppid
        pid  = Process.pid
        sem  = Mutex.new
        $stdout.sync = true
        $stdout.puts "\t#{ppid}.#{pid} STARTED"
        threads = []
        File.open (outpath, 'a+') { |file|
  	file.sync = true
  	n_thread.times {
  	  Thread.new {
  	    sem.synchronize {
  	      tid = threads.size
  	      threads << Thread.current
  	      $stdout.puts "\t\t#{ppid}.#{pid}.#{tid} STARTED"
  	    }
  	    n_mark_per_thread.times {
  	      sem.synchronize {
  		catch (:flock_eintr) {
  		  begin
  		    while (not (file.flock File::LOCK_EX | File::LOCK_NB)) do
  		      Thread.pass
  		    end
  		  rescue
  		    throw :flock_eintr
  		  end
  		}
  		catch (:syswrite_eintr) {
  		  begin
  		    file.syswrite "%10.5f\n" % Time.now.to_f
  		  rescue
  		    throw :syswrite_eintr
  		  end
  		}
  		catch (:flush_eintr) {
  		  begin
  		    file.flush
  		  rescue
  		    throw :flush_eintr
  		  end
  		}
  		catch (:flock_eintr) {
  		  begin
  		    file.flock File::LOCK_UN
  		  rescue
  		    throw :flock_eintr
  		  end
  		}
  	      }
  	      Thread.pass
  	    }
  	  }
  	}
  	threads.each {|t| t.join}
        }
      }
     cids << cid
  }

  cids.each { |cid|
    #Process.waitpid cid
    Process.waitpid cid, Process::WUNTRACED
  }
  cids.each { |cid|
    $stdout.puts "\t#{pid}.#{cid} FINISHED"
  }

  system 'sync'

  output = IO.readlines outpath
  sorted = output.sort {|a,b| a.to_f <=> b.to_f}

  if output != sorted
    File.open (sortedpath, 'w') {|f| f.puts sorted }
    $stdout.puts "\nERROR!"
    $stdout.puts "SUGGEST 'diff -c #{outpath} #{sortedpath}'"
  else
    $stdout.puts "\nSUCCESS!"
  end
----CUT----

-- 

 ====================================
 | Ara Howard
 | NOAA Forecast Systems Laboratory
 | Information and Technology Services
 | Data Systems Group
 | R/FST 325 Broadway
 | Boulder, CO 80305-3328
 | Email: ahoward@fsl.noaa.gov
 | Phone:  303-497-7238
 | Fax:    303-497-7259
 ====================================