Re: Forward references?

"Robert Klemme" <[e-mail]> writes:

> "Lloyd Zusman" <[e-mail]> schrieb im Newsbeitrag
> news:[e-mail]...
>>
>> [ ... ]
>>
>> I refactored the code even more, based on our discussions and some ideas
>> of my own.  If you're interested, I can privately email you the latest
>> version.
>
> [x] interested  [ ] not interested

Here it is.  Let me know what you think.  And thank you for the useful
and interesting discussion in the mailing list.

#!/usr/local/bin/ruby

# Do a 'tail -f' simultaneously multiple files, interspersing their
# output.  Continue tailing any file that has been replaced by a new
# version, as in the following, over-simplified example:
#
#   while :
#   do
#     something >>something.log &
#     pid=$!
#     # ... time passes ...
#     rm -f something.log.old
#     mv something.log something.log.old
#     kill $pid
#   done
#
# See the 'usage' routine, below, for a description of the command
# line options and arguments.

require 'sync'
require 'getoptlong'

$program = File.basename($0)

$stdout.extend(Sync_m)
$stdout.sync = 1

$stderr.sync = 1

$waitTime = 0.25

$defColumns = 80
$defLines   = 80

$maxBlocksize = 1024

# Default values for flags that are set via the command line.
$tailf       = true
$fnamePrefix = false

$opts = GetoptLong.new(
  [ "--lines", "-l", GetoptLong::REQUIRED_ARGUMENT ],
  [ "--exit",  "-x", GetoptLong::NO_ARGUMENT       ],
  [ "--name",  "-n", GetoptLong::NO_ARGUMENT       ],
  [ "--help",  "-h", GetoptLong::NO_ARGUMENT       ]
)

# My list of threads.
$fileThreads = [].extend(Sync_m)

# Main routine
def rtail

  # Calculate the size of a screen so we can choose a reasonable
  # number of lines to tail.

  screenColumns = (ENV['COLUMNS'] == nil ? $defColumns : ENV['COLUMNS']).to_i
  if screenColumns < 1 then
    # In case ENV['COLUMNS'] was set to something <= 0
    screenColumns = $defColumns
  end

  screenLines = (ENV['LINES'] == nil ? $defLines : ENV['LINES']).to_i
  if screenLines < 1 then
    # In case ENV['LINES'] was set to something <= 0
    screenLines = $defLines
  end

  # One more full line than the maximum that the screen can hold ...

  $backwards = screenColumns * (screenLines + 1)


  # Parse and evaluate command-line options.  Temporarily change
  # $0 to be the basename prepended by a newline so that the error
  # message that GetoptLong outputs looks good in the case where
  # an invalid option was entered.

  oldDollar0 = $0
  $0 = "\n" + $program

  begin

    $opts.each do

      |opt, arg|

      case opt
      when "--exit"
	$tailf = false
      when "--lines"
	screenLines = arg.to_i + 0
      when "--name"
	$fnamePrefix = true
      when "--help"
	usage
	# notreached
      else
	usage
	# notreached
      end
    end

  rescue
    usage
    # notreached
  ensure
    $0 = oldDollar0 # just in case we need $0 later on
  end

  if ARGV.length < 1 then
    usage
    # notreached
  end

  # Signal handler.
  [ 'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGTERM' ].each {
    |sig|
    trap(sig) {
      abortThreads(Thread.list.reject {
	|t|
	t == Thread.main
      })
      raise "\n!!! aborted"
      # notreached
    }
  }

  # Start a thread to tail each file whose name appears on
  # the command line.  The threads for any file that cannot
  # be opened for reading will die and will be reaped in
  # the main loop, below.
  ARGV.each {
    |arg|
    $fileThreads.synchronize {
      $fileThreads << Thread.new(arg, $tailf, &$fileReadProc)
    }
  }

  # Main loop: reap dead threads and exit once there are no more
  # threads that are alive.
  loop {
    tcount = 0
    $fileThreads.synchronize {
      tcount = $fileThreads.length
    }
    if tcount < 1 then
      break
    else
      # Don't eat up too much of my CPU time
      waitFor($waitTime)
    end
  }

  # Bye-bye
  return 0
end

# This is a mixin for adding a textfile? method to a class that
# behaves like IO.  It also adds an externally callable 
# TextTester.text? method to test a block of data.

module TextTester

private
  # List of items that I want to treat as being normal text
  # characters.  The first line adds a lot of European characters
  # that are not normally considered to be text characters in
  # the traditional routines that distinguish between text and
  # binary files.  This is used within the 'textfile?' method.
  @@textpats = [ "^瘟逑洄蓐鴒粳阮ⅷ苟蘇羊收般燮痛榴廃／",
                 "^ -~",
                 "^\b\f\t\r\n" ]

public

  # This is my own, special-purpose test for text-ness.  I don't want to
  # treat certain European characters as binary.  If the 'testsize'
  # argument is non-nil, try to read a buffer of that size; otherwise,
  # calculate the buffer size here.  If the 'restorePosition' argument
  # is true, make sure that the the position pointer within the IO
  # handle gets repositioned back to its initial value after this test
  # is performed.
  #
  # This method is callable directly from outside the module.  Hence,
  # I define it as self.text? here.

  def self.text?(block, len = nil)
    if len.nil? then
      len = block.length
    end
    return (block.count(*@@textpats) < (len / 3.0) and block.count("\x00") < 1)
  end

  def textfile?(testsize = nil, restorePosition = true)
    begin
      if restorePosition then
	pos = self.pos
      else
	pos = nil
      end
      if testsize.nil? then
	testsize = [ self.stat.blocksize,
	             self.stat.bytesize,
	             $maxBlocksize ].min
      end
      block = self.read(testsize)
      len = block.length
      if len < 1 then
	return true # Provisionally treat a zero-length file as a text file.
      end

      # I need to call text? both inside and outside of this module.
      # Therefore, I have to define that method as self.text?, which
      # requires me to explicitly reference it off of TextTester here.
      result = TextTester.text?(block, len)
      unless pos.nil?
	self.seek(pos, IO::SEEK_SET)
      end
      return result
    rescue
      return false
    end
  end
end

# Add the test for a text file into the IO class.

class IO
  include TextTester
end


# Do a timed 'wait'.

def waitFor(duration)
  startTime = Time.now.to_f
  select(nil, nil, nil, duration)
  Thread.pass
  # We could be back here long before 'duration' has passed.
  # The loop below makes sure that we wait at least as long
  # as this specified interval.
  while (elapsed = (Time.now.to_f - startTime)) < duration
    select(nil, nil, nil, 0.001)
    Thread.pass
  end
  # Return the actual amount of time that elapsed.  This is
  # guaranteed to be >= 'duration'.
  return elapsed
end


# We make sure that $stdout is synchronized so that lines of
# data coming from different threads don't garble each other.

def syncwrite(text)
  begin
    $stdout.synchronize(Sync::EX) {
      $stdout.write(text)
    }
  rescue
    # Fall back to normal, non-sync writing
    $stdout.write(text)
  end
end


# Decide whether to output a block as is, or with a prefix
# at the beginning of each line.  In the "as is" case, just
# send the whole block to 'syncwrite'; otherwise, split into
# lines and prepend the prefix before outputting.  In other
# words, we only incur the cost of splitting the block when
# we absolutely have to.

def output(item)
  prefix, block = item
  if prefix.nil? or prefix.length < 1 then
    syncwrite(block)
  else
    block.split(/\r*\n/).each {
      |line|
      syncwrite(prefix + line + "\n")
    }
  end
end


# Remove a group of threads from the list and kill each one.

def abortThreads(tlist)
  $fileThreads.synchronize {
    tlist.each {
      |t|
      $fileThreads.delete(t)
      t.kill
    }
  }
end


# Remove myself from the thread list and kill myself.
def abortMyself
  abortThreads([Thread.current])
  # notreached
end


# Close the specified IO handle and kill the containing thread
# if this fails.

def closeOrDie(f)
  begin
    f.close()
  rescue
    output([nil, "!!! unable to close file: #{item}\n"])
    abortMyself()
    # notreached
  end
end


# This is the main thread proc for tailing a given file.

$fileReadProc = Proc.new do
  |item, follow|

  # Open the file, make sure it's a text file, read the last bit
  # at the end, and output it.  Kill the containing thread if any
  # of this fails.

  begin
    f = File.open(item, 'r')
  rescue
    output([nil, "!!! unable to open: #{item}\n"])
    abortMyself()
    # notreached
  end

  # Get some info about the open file
  begin
    f.sync = true
    bytesize = f.stat.size
    blocksize = f.stat.blksize
    inode = f.stat.ino
  rescue
    f.close
    output([nil, "!!! unable to stat: #{item}\n"])
    abortMyself()
    # notreached
  end

  # Blocksize will be nil or zero if the device being opened
  # is not a disk file.  Bytesize will also be nil in this case.
  if blocksize.nil? or blocksize < 1 or bytesize.nil? then
    f.close
    output([nil, "!!! invalid device: #{item}\n"])
    abortMyself()
    # notreached
  end

  # Test for text-ness using one blocksize unit, or the length
  # of the file if that is smaller.  This is done in two statements
  # because we need to use 'blocksize' by itself, further down in
  # this procedure.
  blocksize = [ blocksize, $maxBlocksize ].min
  testsize = [ blocksize, bytesize ].min
  unless f.textfile?(testsize, false) then
    f.close
    output([nil, "!!! not a text file: #{item}\n"])
    abortMyself()
    # notreached
  end

  # Set the optional output line prefix.
  if $fnamePrefix then
    prefix = File.basename(item) + ': '
  else
    prefix = nil
  end

  textTestLength = 0

  # Position to a suitable point near the end of the file,
  # and then read and output the data from that point until
  # the end.
  begin
    if bytesize > $backwards then
      pos = bytesize - $backwards
    else
      pos = 0
    end
    f.seek(pos, IO::SEEK_SET)
    if pos > 0 then
      f.gets # discard possible line fragment
    end
    readSoFar = f.read
    textTestLength = readSoFar.length
    output([prefix, readSoFar])
  rescue
  end

  # If we have made it here, we've read the last bit of the file
  # and have output it.  Now, if we're not in 'follow' mode, we
  # just exit.
  unless follow then
    f.close
    abortMyself()
    # notreached
  end

  # We only arrive here if we're in 'follow' mode.  In this case,
  # we keep looping to test if there is any more data to output.
  loop {
    #
    # The file might have been closed due to it having disappeared
    # or having changed names.  If so, reopen it.
    #
    if f.closed? then
      begin
	f = File.open(item, 'r')
	f.sync = true
	textTextLength = 0
	readSoFar = ''
	inode = f.stat.ino
	output([nil, "!!! reopened: #{item}\n"])
	# Fall through to the EOF test.
      rescue
	output([nil, "!!! disappeared: #{item}\n"])
	begin
	  f.close
	rescue
	end
	abortMyself()
	# notreached
      end
    else # file is not closed
      #
      # File was not previously closed, so we can test to see if it
      # has changed or disappeared.
      #
      # Get the current inode of the file.  This is needed to test
      # whether or not the file has disappeared and whether or not there
      # is a new file by the same name.  This is not 100-percent
      # conclusive, since a new file might accidentally end up with the
      # same inode of an older, deleted file.
      #
      begin
	newinode = File.stat(item).ino
	# Fall through to the EOF test.
      rescue
	# If we're here, the file has disappeared.  Close the handle,
	# wait a bit, and then try to reopen it.
	closeOrDie(f)
	waitFor($waitTime)
	redo # go back and iterate again
        # notreached
      end

      if newinode != inode then
	# If we're here, the file was replaced by a new file of 
	# the same name.  Close the handle, wait a bit, and then
	# try to reopen it.
	closeOrDie(f)
	waitFor($waitTime)
	redo # go back and iterate again
        # notreached
      end

    end # f.closed? ... else ...

    # The only way that we can get to this point is if the file is
    # properly open and it hasn't been deleted or replaced.

    if f.eof? then
      # If we're here, we're at EOF.  Reset the EOF indicator and
      # try again.
      f.seek(0, IO::SEEK_CUR)
      waitFor($waitTime)
      redo # go back and iterate again
      # notreached
    end

    # If we're here, we're not at EOF.

    if f.pos < f.stat.size then

      # If we're here, more data was added to the file since the last
      # time we checked.  Output this data, relinquish control to
      # other threads, and then repeat the loop.
      #
      # If we haven't yet tested a full block's worth of bytes
      # for text-ness, continue that test here.

      data = f.read
      if textTestLength < blocksize then
	len = data.length
	textTestLength += len
	readSoFar << data
	if len > 0 and not TextTester.text?(readSoFar) then
	  # If we're here, it's not a text file after all.
	  closeOrDie(f)
	  output([nil, "!!! not a text file: #{item}\n"])
	  abortMyself()
	  # notreached
	end
      end
      output([prefix, data])
      Thread.pass
      redo # go back and iterate again
      # notreached
    end

    # If we're here, the file hasn't changed since last time.
    # Wait a bit so as to not eat up too much CPU time.

    waitFor($waitTime)

  } # end of loop

end # end of thread proc


# Print a usage message and exit.
def usage
  raise <<EOD

usage: #{$program} [ options ] file [ ... ]

options:

  --help, -h            print this usage message

  --lines=<n>, -l <n>   tail <n> lines of each file (default #{$defLines})

  --exit, -x            exit after showing initial tail

  --name, -n            prepend file basename on each line that is output

EOD
  # notreached

end


# Run it
begin
  result = rtail
rescue Exception => e
  $stderr.puts(e)
  result = 1
end
exit(result)

__END__


-- 
 Lloyd Zusman
 [e-mail]
 God bless you.