Re: Rubicon -> Test::Unit [long]

Bil Kleb wrote:
> 
> Are there plans a foot for Rubicon to use Test::Unit?
> 
> I ask because we currently use Rubicon's BulkTestRunner
> to summarize our RubyUnit test results.

So, apparently being too impatient for an answer, I stole the
pertinent bits from rubicon, somehow(?!) modified them to
work with Test::Unit (including adding attr_readers for failures
and errors to Test::Unit::TestResult), and created a stand-alone
BulkTestRunner which runs all the tests in all the files that
it is passed and produces a rubicon-style summary, e.g.,

 ========================================================================
 Mobility Test Summary         Test Results
 ========================================================================
                  Name   OK?   Tests  Asserts      Failures   Errors
             ------------------------------------------------------------
             Array2DUT             3        6
     BoundaryProfileUT             7       31
                 .                 .        .
             PhysicsUT             3        8
              VectorUT             6        7
 ========================================================================
          All 17 files            80      231          0        0
 ========================================================================

This was produced by the following script, run in a directory containing
a bunch of files where files containing UnitTests all ending with "UT.rb".

 <file name="RunAllTests.rb">
  #!/usr/bin/env ruby
  require 'test/unit/ui/console/bulktestrunner'
  tests = Test::Unit::UI::Console::BulkTestRunner.new([],"Mobility Test Summary")
  Dir["*UT.rb"].each{|ut| tests.addFile(ut)}
  tests.run
 </file>

Here's bulktestrunner.rb for those crazy enough to still be reading along...

# $RCSfile: bulktestrunner.rb,v $

# An incarnation of Dave Thomas's rubicon BulkTestRunner
# (and associated ascii summary output) adapted for use
# with Test::Unit

# $Mangler: Bil Kleb <[e-mail]> $
# $Date: 2002/03/20 03:08:39 $
# $Revision: 1.1 $

require 'test/unit/ui/console/testrunner'

module Test
 module Unit
  module UI
   module Console

    # Run sets of tests in multiple files. This would be a TestSuite,
    # but we want to run each file separately, and to summarize the
    # results differently.

    class BulkTestRunner

     def initialize(args, group_name='')
      @files     = Array.new
      @results   = ResultGatherer.new
      @results.name = group_name
     end

     # Push a file onto the array of files to process.
     def addFile(fileName)
      @files.push fileName
     end

     # Invoke the TestRunner for each file and collect the results.
     # Finally, produce the report.
     def run
      @files.each do |file|
       require file
       className = File.basename(file)
       className.sub!(/\.rb$/, '')
       klass = eval className
       runner = Test::Unit::UI::Console::TestRunner.new(klass.suite, true)
       $stderr.print "\n", className, ": "
       @results.add(klass, runner.start)
      end

      reporter = ResultDisplay.new(@results)
      reporter.reportOn $stdout
      @results.failure_count
     end

    end # class BulkTestRunner

    # This is where we gather the results of all the tests.

    class ResultGatherer

     attr_accessor :name
     attr_reader   :results
     attr_reader   :failure_count

     def initialize(name='')
      @name = name
      @results = Hash.new
      @failure_count = 0
     end

     def add(klass, result_set)
      @results[klass.name] = Results.new.initialize_from(result_set)
      @failure_count += result_set.error_count + result_set.failure_count
     end
     
    end # class ResultGatherer

    # Objects of this class get generated from the TestResult
    # passed back by Test::Unit. We don't use it's class for two reasons:
    # 1. We de-couple better this way
    # 2. We can't serialize the Test::Unit class, as it contains IO objects
    
    class Results

     attr_reader :errors
     attr_reader :failures
     attr_reader :run_count
     attr_reader :error_count
     attr_reader :failure_count
     attr_reader :assertion_count

     def initialize_from(test_result)
      @errors           = Failure.from_real_failures(test_result.errors)
      @failures         = Failure.from_real_failures(test_result.failures)
      @run_count        = test_result.run_count
      @error_count      = test_result.error_count
      @failure_count    = test_result.failure_count
      @assertion_count  = test_result.assertion_count
      @passed           = test_result.passed?
      self
     end

     def passed?
      @passed
     end

    end # class Results

    # Record a particular failure, which is a location
    # and an error message. We simply ape the Test::Unit
    # TestFailure class.

    class Failure

     attr_accessor :location
     attr_accessor :message
     
     def Failure.from_real_failures(f)
      f.collect do |a_failure|
       my_f = Failure.new
       my_f.location  = a_failure.location
       my_f.message = a_failure.message
       my_f
      end
     end

    end # class Failure

    # Produce a report, given results.
    
    class ResultDisplay

     LINE_LENGTH = 72
     LINE = '=' * LINE_LENGTH
     Line = ' ' * 12 + '-' * (LINE_LENGTH - 12)
     
     def initialize(gatherer)
      @results = gatherer.results
      @name    = gatherer.name
     end

     # Write a report to the "op" file handle
     def reportOn(op)
      op.puts
      op.puts LINE
      title = "Test Results".center(LINE_LENGTH)
      title[0, @name.length] = @name
      op.puts title
      op.puts LINE
      op.puts "                 Name   OK?   Tests  Asserts      Failures   Errors"
      op.puts Line    
      total_classes = 0
      total_tests   = 0
      total_asserts = 0
      total_fails   = 0
      total_errors  = 0
      total_bad     = 0
      
      format = "%21s   %4s   %4d  %7d  %9s  %7s\n"
      
      names = @results.keys.sort
      for name in names
       res    = @results[name]
       fails  = res.failure_count.nonzero? || ''
       errors = res.error_count.nonzero?   || ''
       
       total_classes += 1
       total_tests   += res.run_count
       total_asserts += res.assertion_count
       total_fails   += res.failure_count
       total_errors  += res.error_count
       total_bad     += 1 unless res.passed?
       
       op.printf format,
	name.sub(/^Test/, ''),
	res.passed? ? "    " : "FAIL",
	res.run_count, res.assertion_count, 
	fails.to_s, errors
      end
      
      op.puts LINE
      if total_classes > 1
       op.printf format, 
	sprintf("All %d files", total_classes),
	total_bad > 0 ? "FAIL" : "    ",
	total_tests, total_asserts,
	total_fails, total_errors
       op.puts LINE
      end
      
      if total_fails > 0
       op.puts
       op.puts "Failure Report".center(LINE_LENGTH)
       op.puts LINE
       left = total_fails
       
       for name in names
	res = @results[name]
	if res.failure_count > 0
	 op.puts
	 op.puts name + ":"
	 op.puts "-" * name.length.succ
	 
	 res.failures.each do |f|
	  f.location.each do |at|
	   break if at =~ /rubicon/
	   op.print "    ", at, "\n"
	  end
	  err = f.message.to_s
	  
	  if err =~ /expected:(.*)but was:(.*)/m
	   exp = $1.dump
	   was = $2.dump
	   op.print "    ....Expected: #{exp}\n"
	   op.print "    ....But was:  #{was}\n"
	  else
	   op.print "    ....#{err}\n"
	  end
	 end
	 
	 left -= res.failure_count
	 op.puts
	 op.puts Line if left > 0
	end
       end
       op.puts LINE
      end
      
      if total_errors > 0
       op.puts
       op.puts "Error Report".center(LINE_LENGTH)
       op.puts LINE
       left = total_errors
       
       for name in names
	res = @results[name]
	if res.error_count > 0
	 op.puts
	 op.puts name + ":"
	 op.puts "-" * name.length.succ
	 
	 res.errors.each do |f|
	  f.location.each do |at|
	   break if at =~ /rubicon/
	   op.print "    ", at, "\n"
	  end
	  err = f.message.to_s
	  op.print "    ....#{err}\n"
	 end
	 
	 left -= res.error_count
	 op.puts
	 op.puts Line if left > 0
	end
       end
       op.puts LINE
      end
     end # def reportOn

    end # class ResultDisplay

   end # module Console
  end # module UI
 end # module Unit
end # module Test

Just to be complete and *extra long*, here is a sample UnitTest file,

 <file name="CellUT.rb">
  require 'test/unit'
  require 'Cell'

  class CellUT < Test::Unit::TestCase
   def set_up
    @state    = 0.2
    @aCell    = Cell.new(@state)
   end
   def testCreation
    assert_equal(@state,@aCell.state)
    @aCell = Cell.new()
    assert_equal(0, @aCell.state)
    assert_equal(0,@aCell.deltaUi)
    assert_equal(0,@aCell.deltaUj)
   end
   def testUpdateDisabled
    assert_equal(0,@aCell.update)
   end
  end
 </file>

-- 
Bil Kleb
NASA Langley Research Center
Hampton, Virginia, USA