From: Lloyd Zusman <ljz@...>
Date: 2004-08-01T02:21:51+09:00
Subject: Re: Forward references?

--=-=-=

"Robert Klemme" <bob.news@gmx.net> writes:

> "Lloyd Zusman" <ljz@asfast.com> schrieb im Newsbeitrag
> news:m3n01g5p5c.fsf@asfast.com...
>>
>> [ ... ]
>>
>> I just want to have my program's main body at the top of the program
>> file, and the subsidiary functions at the end.  The only thing C-like
>> about this is the fact that I chose the word "main" for the routine that
>> houses the main body of code.
>>
>> This construct is not necessary.  I could just as easily name the main
>> routine as "foobar", and it would look less C-like without losing the
>> fact that the main body of the code comes first in the program file.
>
> Here's another idea: put your code into two files and require (or load) the
> helper code.

I know that I can do that.  But often I just want one file.  I believe
that the best use of `require' or `load' is to include code from shared
libraries.  Putting simple subsidiary routines in one or more separate
files often complicates installation and maintenance.


> Or put your helper code after __END__ and use eval to compile it:
>
> [ ... etc. ... ]

That can work, but if I ever want to put real data after __END__ and
read it via the DATA handle, I'd be out of luck.


>> Never fear ... the code that I write tends to look ruby-ish and not
>> C-like. :)
>
> Dare you!  :-)

OK.  Attached is a ruby program that I recently wrote.  It is a
specialized "tail -f".  In addition to standard "tail -f" capabilities,
it also can simultaneously tail multiple files, and in addition, it will
detect if a new file has replaced the one that is being tailed, in which
case it starts tailing the newly created file automatically.

The second feature is useful when I'm tailing log files that get
recycled.  For example, if syslog always writes log data to a file
called, say, "foobar.log", and if once a day the following commands are
run ...

  /bin/rm -f foobar.log.old
  /bin/mv foobar.log foobar.log.old
  kill -HUP $pid # where $pid is the process ID for syslogd

... then after these commands are invoked, syslog will start writing
log data to a new, empty "foobar.log" file.  If I had done this ...

  tail -f foobar.log

... then I would keep looking at the no-longer-changing "foobar.log.old"
file after the log files are recycled.  But if I invoke my new command
(which I call "rtail") as follows ...

  rtail foobar.log

... then once the logs are recycled, the data that is being added to
"foobar.log" will continue to appear in real time.

Here's the code ...


--=-=-=
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=rtail
Content-Transfer-Encoding: quoted-printable
Content-Description: Ruby program 'rtail'

#!/usr/bin/ruby

# Do a 'tail -f' simultaneously multiple files, interspersing their
# output.
#
# See the 'usage' routine, below, for a description of the command
# line options and arguments.

require 'sync'
require 'getoptlong'

$program =3D $0.sub(/^.*\//, '')

$stdout.extend(Sync_m)
$stdout.sync =3D 1

$stderr.sync =3D 1

$waitTime    =3D 0.25

# Default values for flags that are set via the command line.
$tailf       =3D true
$fnamePrefix =3D false

$defLineLen  =3D 80
$defLines    =3D 80
$lineLen     =3D (ENV['COLUMNS'] =3D=3D nil ? $defLineLen : ENV['COLUMNS'])=
.to_i
if $lineLen < 1 then
  $lineLen   =3D $defLineLen
end
$lines       =3D (ENV['LINES'] =3D=3D nil ? $defLines : ENV['LINES']).to_i

$opts =3D GetoptLong.new(
  [ "--help",  "-h", GetoptLong::NO_ARGUMENT       ],
  [ "--lines", "-l", GetoptLong::REQUIRED_ARGUMENT ],
  [ "--exit",  "-x", GetoptLong::NO_ARGUMENT       ],
  [ "--name",  "-n", GetoptLong::NO_ARGUMENT       ]
)

# My list of threads.
$fileThreads =3D []

# Main routine
def rtail

  begin

    # Parse and evaluate command-line options
    $opts.each do

      |opt, arg|

      case opt
      when "--help"
        usage
	# notreached
      when "--lines"
        begin
          $lines =3D arg.to_i + 0
        rescue
          usage
	  # notreached
        end
      when "--exit"
        $tailf =3D false
      when "--name"
        $fnamePrefix =3D true
      end
    end
  rescue
    usage
    # notreached
  end

  if ARGV.length < 1 then
    usage
    # notreached
  end

  # Calculate the size of a screen so we can choose a reasonable
  # number of lines to tail.
  if $lines < 1 then
    $lines =3D $defLines
  end

  # One more full line than the maximum that the screen can hold.
  $backwards =3D $lineLen * ($lines + 1)

  # Signal handler.
  [ 'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGTERM' ].each {
    |sig|
    trap(sig) {
      Thread.critical =3D true
      Thread.list.each {
	|t|
	unless t =3D=3D Thread.main then
	  t.kill
	end
      }
      $stderr.puts("\r\n!!! aborted")
      Thread.critical =3D false
      exit(-1)
    }
  }

  # Start a thread to tail each file whose name appears on
  # the command line.  The threads for any file that cannot
  # be opened for reading will die and will be reaped in
  # the main loop, below.
  ARGV.each {
    |arg|
    Thread.critical =3D true
    $fileThreads << Thread.new(arg, $tailf, &$fileReadProc)
    Thread.critical =3D false
  }

  # Main loop: reap dead threads and exit once there are no more
  # threads that are alive.
  loop {
    Thread.critical =3D true
    tcount =3D $fileThreads.length
    Thread.critical =3D false
    if tcount < 1 then
      break
    else
      # Don't eat up too much of my CPU time
      waitFor($waitTime)
    end
  }

  # Bye-bye
  exit(0)
end

# Add my own 'textfile?' method to the IO class.
class IO

private
  # List of items that I want to treat as being normal text
  # characters.  The first line adds a lot of European characters
  # that are not normally considered to be text characters in
  # the traditional routines that distinguish between text and
  # binary files.  This is used within the 'textfile?' method.
  @@textpats =3D [ "^=E1=E9=ED=F3=FA=E0=E8=EC=F2=F9=E4=EB=EF=F6=FC=F8=E7=F1=
=C1=C9=CD=D3=DA=C0=C8=CC=D2=D9=C4=CB=CF=D6=DC=D8=C7=D1=A1=BF",
                 "^ -~",
                 "^\b\f\t\r\n" ]

public
  # This is my own, special-purpose test for text-ness.  I don't want
  # to treat certain European characters as binary.  If the
  # 'restorePosition' argument is true, make sure that the the position
  # pointer within the IO handle gets repositioned back to its initial
  # value after this test is performed.
  def textfile?(testsize, restorePosition =3D false)
    if restorePosition then
      pos =3D self.pos
    else
      pos =3D nil
    end
    begin
      block =3D self.read(testsize)
    rescue
      return false
    end
    len =3D block.length
    if len < 1 then
      return true # I want to treat a zero-length file as a text file.
    end
    result =3D (block.count(*@@textpats) < (len / 3.0) and=20
	      block.count("\x00") < 1)
    unless pos.nil?
      begin
        self.seek(pos, IO::SEEK_SET)
      rescue
        return false
      end
    end
    return result
  end
end


# Do a timed 'wait'.
def waitFor(duration)
  startTime =3D Time.now.to_f
  select(nil, nil, nil, duration)
  Thread.pass
  # We could be back here long before 'duration' has passed.
  # The loop below makes sure that we wait at least as long
  # as this specified interval.
  while (elapsed =3D (Time.now.to_f - startTime)) < duration
    select(nil, nil, nil, 0.001)
    Thread.pass
  end
  # Return the actual amount of time that elapsed.  This is
  # guaranteed to be >=3D 'duration'.
  return elapsed
end

# We make sure that $stdout is syncrhonized so that lines of
# data coming from different threads don't garble each other.
def syncwrite(text)
  begin
    $stdout.synchronize(Sync::EX) {
      $stdout.write(text)
    }
  rescue
    # Fall back to normal, non-sync writing
    $stdout.write(text)
  end
end

# Decide whether to output a block as is, or with a prefix
# at the beginning of each line.  In the "as is" case, just
# send the whole block to `syncwrite'; otherwise, split into
# lines and prepend the prefix before outputting.  In other
# words, we only incur the cost of splitting the block when
# we absolutely have to.
def output(item)
  prefix, block =3D item
  if prefix.nil? or prefix.length < 1 then
    syncwrite(block)
  else
    block.split(/\r*\n/).each {
      |line|
      syncwrite(prefix + line + "\n")
    }
  end
end

# Remove myself from the thread list and kill myself.
def abortMyself
  t =3D Thread.current
  Thread.critical =3D true
  $fileThreads.delete(t)
  Thread.critical =3D false
  t.kill
end

# The main thread proc for tailing a given file
$fileReadProc =3D Proc.new do
  |item, follow|

  # Open the file, make sure it's a text file, read the last bit
  # at the end, and output it.  Kill the containing thread if any
  # of this fails.
  begin
    f =3D File.open(item, 'r')
  rescue
    output([nil, "!!! unable to open: #{item}\n"])
    abortMyself()
  end
  # Get some info about the open file
  begin
    f.sync =3D true
    bytesize =3D f.stat.size
    blocksize =3D f.stat.blksize
    inode =3D f.stat.ino
  rescue
    f.close
    output([nil, "!!! unable to stat: #{item}\n"])
    abortMyself()
  end
  # Blocksize will be nil or zero if the device being opened
  # is not a disk file.  Bytesize will also be nil in this case.
  if blocksize.nil? or blocksize < 1 or bytesize.nil? then
    f.close
    output([nil, "!!! invalid device: #{item}\n"])
    abortMyself()
  end
  # Test for text-ness using one blocksize unit, or the length
  # of the file if that is smaller.
  testsize =3D (blocksize < bytesize ? blocksize : bytesize)
  unless f.textfile?(testsize) then
    f.close
    output([nil, "!!! not a text file: #{item}\n"])
    abortMyself()
  end
  if $fnamePrefix then
    prefix =3D File.basename(item) + ': '
  else
    prefix =3D nil
  end

  # Position to a suitable point near the end of the file,
  # and then read and output the data from that point until
  # the end.
  begin
    if bytesize > $backwards then
      pos =3D bytesize - $backwards
    else
      pos =3D 0
    end
    f.seek(pos, IO::SEEK_SET)
    if pos > 0 then
      f.gets # discard possible line fragment
    end
    output([prefix, f.read])
  rescue
  end

  # If we have made it here, we've read the last bit of the file
  # and have output it.  Now, if we're not in 'follow' mode, we
  # just exit.
  unless follow then
    f.close
    abortMyself()
  end

  # We can only be here if we're in 'follow' mode.  In this case,
  # we keep looping to test if there is any more data to output.
  loop {
    # Get the current inode of the file.  This is used to test whether
    # or not the file has disappeared and whether or not there is a
    # new file by the same name.  This is not 100-percent conclusive,
    # since a new file might accidentally end up with the same inode
    # of an older, deleted file.
    begin
      newinode =3D File.stat(item).ino
    rescue
      newinode =3D nil
    end
    begin
      if newinode.nil? or newinode !=3D inode then
	# If we're here, the file disappeared or was replaced by
	# a new file of the same name.  We try to reopen the new
	# version before continuing with the loop.
	begin
	  f.close
	  waitFor($waitTime) # Wait a bit before trying to reopen
	  f =3D File.open(item, 'r')
	  f.sync =3D true
	  unless f.textfile?(testsize, true) then
	    f.close
	    output([nil, "!!! reopenable, but not a text file: #{item}\n"])
	    abortMyself()
	  end
	  inode =3D newinode
	  output([nil, "!!! reopened: #{item}\n"])
	rescue
	  output([nil, "!!! disappeared: #{item}\n"])
	  begin
	    f.close
	  rescue
	  end
	  abortMyself()
	end
      elsif f.eof? then
	# If we're here, we're at EOF.
	f.seek(0, IO::SEEK_CUR)
	waitFor($waitTime)
      elsif f.pos < f.stat.size then
	# If we're here, more data was added to the file since the last
	# time we checked.  Output this data, relinquish control to
	# other threads, and then repeat the loop.
	output([prefix, f.read])
	Thread.pass
      else
	# If we're here, the file hasn't changed since last time.
	# Wait a bit so as to not eat up too much CPU time.
	waitFor($waitTime)
      end
    rescue
      # Can we ever get here?
    end
  } # end of loop.
end # end of thread proc

# Print a usage message and exit.
def usage
  warn <<EOD

usage: #{$program} [ options ] file [ ... ]

options:

  --help, -h            print this usage message

  --lines=3D<n>, -l <n>   tail <n> lines of each file (default #{$defLines})

  --exit, -x            exit after showing initial tail

  --name, -n            prepend file basename on each line that is output
EOD
  exit(1)
end

# Run it
exit(rtail)

__END__

--=-=-=



-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

--=-=-=--