From: ahoward <ahoward@...>
Date: 2003-01-27T12:56:45+09:00
Subject: Re: solaris porting problem -- flock failure?

On Sun, 26 Jan 2003, Joel VanderWerf wrote:

> I'm having trouble porting some multi-thread, multi-process code from
> linux to solaris. I've already dealt with (or tried to deal with) some
> differences in flock (solaris flock is based on fcntl locks), like the
> fact that closing a file releases locks on the file held by other threads.
>
> I've managed to isolate the problem in a fairly simple test program. It's at
>
>    http://path.berkeley.edu/~vjoel/ruby/solaris-bug.rb

here is a similar program, which works on both linux :

  /usr/home/howardat/eg/ruby > uname -srpm
  Linux 2.2.14-5.0 i686 unknown
  /usr/home/howardat/eg/ruby > ruby -v
  ruby 1.6.5 (2001-10-22) [i686-linux]

and solaris

  > uname -srpm
  SunOS 5.8 sun4u sparc
  > ruby -v
  ruby 1.6.6 (2001-12-26) [sparc-solaris2.8]


the concept of the program is that mutiple processes spawn multiple threads,
each of which attempt to gain and exclusive lock on a file in order to mark a
time stamp.  when all process have waited for all threads, and all child
process themselves have been waited for, the file is read back in and sorted.
if any entries are found to be out of order - there was an error.

espcially note the need to retry (catch/throw) in the event of interupted
system calls.  this is not needed on linux, but it happens _alot_ on solaris.
i do not know why.  calling "system 'sync'" is _absolutely_ essentially
otherwise kernel buffers will still have data in them when the file is read
back in (yes even with "file.sync = true"!).  i can't believe how painfull this
was to do... if anyone knows any short-cuts i'd be more than happy to hear
them.  also, if anyone could run this on their platform that'd be great

file : feeding_frenzy.rb

----CUT----

  #!/usr/bin/env ruby

  require 'thread'
  require 'ftools'

  n_process = (ARGV.shift or 2).to_i
  n_thread  = (ARGV.shift or 2).to_i
  n_mark    = (ARGV.shift or 2 << 9).to_i
  out       = (ARGV.shift or 'out')


  File.rm_f out
  pids = []

  n_process.times {

    pid =
      Process.fork {
	sem = Mutex.new
	File.open (out, 'a+') { |file|
	  file.sync = true
	  n_thread.times {
	    t =
	      Thread.new {
		(n_mark / n_thread).times {
		  sem.synchronize {
		    catch (:flock_eintr) {
		      begin
			while ((file.flock File::LOCK_EX | File::LOCK_NB)) do
			  Thread.pass
			end
		      rescue
			puts 'FLOCK EINTR'
			throw :flock_eintr
		      end
		    }
		    catch (:syswrite_eintr) {
		      begin
			file.syswrite "%10.5f\n" % Time.now.to_f
		      rescue
			puts 'SYSWRITE EINTR'
			throw :syswrite_eintr
		      end
		    }
		    catch (:flock_eintr) {
		      begin
			file.flock File::LOCK_UN
		      rescue
			puts 'FLOCK EINTR'
			throw :flock_eintr
		      end
		    }
		  }
		}
	      }
	    t.join
	  }
	}
      }

    pids << pid
  }

  pids.each { |pid|
    Process.waitpid pid, Process::WNOHANG
  }

  system 'sync'

  output = IO.readlines out
  sorted = output.sort {|a,b| a.to_f <=> b.to_f}

  if output != sorted
    puts 'ERROR!'
    File.open 'sorted', 'w' do |f|
      f.puts sorted
    end
  end

----CUT----


-ara


-- 

 ====================================
 | Ara Howard
 | NOAA Forecast Systems Laboratory
 | Information and Technology Services
 | Data Systems Group
 | R/FST 325 Broadway
 | Boulder, CO 80305-3328
 | Email: ahoward@fsl.noaa.gov
 | Phone:  303-497-7238
 | Fax:    303-497-7259
 ====================================