From: Johan Holmberg <holmberg@...>
Date: 2003-09-15T21:08:22+09:00
Subject: Test::Unit -- multiple errors in test method ???


Hi !

I have been writing some unit tests with Test::Unit.

I've noted that when an assertion fails in a test method, the
remaining assertions in the same test method aren't even excuted.
Here is an example:

      require 'test/unit'

      class TC_example < Test::Unit::TestCase

	  def test_a
	      assert_equal 2,  1 + 1
	      assert_equal 5,  2 + 2         # gives error
	      assert_equal 7,  2 + 5         # never executed !!!
	      assert_equal 8,  3 + 4         # never executed !!!
	  end

      end

When running this I get:

      C:\> ruby TC_example.rb
      Loaded suite TC_example
      Started
      F
      Finished in 0.01 seconds.

	1) Failure!!!
      test_a(TC_example) [TC_example1.rb:8]:
      <5> expected but was
      <4>

      1 tests, 2 assertions, 1 failures, 0 errors


The two last assertions aren't even executed.
Is this a but or an intentional "feature" ?

I've looked at the way JUNIT in Java does the same thing,
and there I got *all* errors reported. That seems much more useful
to mee.

With the current Test::Unit behaviour in Ruby, I can't know if
an error "hides" a number of other errors until I've fixed the first
error (it feels like having a C-compiler that only report the first
compilation error ...).

I have tested this with Ruby 1.8, and also with a more recent
CVS-snapshot.

Am I missing something obvious,
or should this be considered a bug in Test::Unit ?

/Johan Holmberg