From: Phlip <phlip_cpp@...>
Date: 2005-04-16T03:24:36+09:00
Subject: Re: Multiple asserts in a single unit test

Daniel Berger wrote:

> Any opinions on multiple asserts within a single unit test?

The distinction is roughly between testing greenfield code with testing
legacy systems.

In greenfield code, test-first can force so much decoupling that you don't
/want/ to assert more than one thing in each case. So setting "one assertion
per case" as a design goal

Some tests stretch the definition of "one assertion":

  def test_def_paren
    tokenize( "def  foo(bar)" )
    assert_next_token :keyword, "def  "
    assert_next_token :method, "foo"
    assert_next_token :punct, "("
    assert_next_token :ident, "bar"
    assert_next_token :punct, ")"
  end

That is an assertion of one variable (a hidden @member), chained out to show
each tested detail in the variable. (It's from Jamis Buck's Syntax module.)

Legacy systems, by contrast, typically test at a higher level. If you test a
Tk GUI, simulating a click on a Tk Canvas might change many configurations.
Rather than waste the test state, and the time required to paint a canvas,
write a list of assertions that check each important configuration.

>  Any
> opinions on what the default behavior should be for test-unit with
> regards to multiple asserts where one fails?

If you have a debugger, such as for VC++, an assertion should raise a
breakpoint, giving a programmer the option to run more lines.

Unattended, some assertions should drop-dead, and some should keep going.
However, assertion systems already have numerous permutations (assert_equal,
assert_match, assert_nil, modulo _not, etc.). Adding another permutation
would blow our minds.

In C++, I use a raw ASSERT() to drop an entire test run dead, and use
CHECK() for relatively recoverable situations. CHECK(), in unattended mode,
keeps going.

Ultimately, the rule for Test-Driven Development is you don't coddle the
test failure. Your rig should provide automated navigation to the failing
assertion, and should reflect its variables and values. You should either
instantly fix the problem or instantly use Undo to run back to the last
failing state. Coddling the test failures - making a list of them, logging
them, e-mailing them to your boss - are all secondary considerations.

So, all assertions could just drop a test run dead, and TDD will work fine.
Your nightly unattended test run will always pass, so recovering from
assertions is irrelevant.

-- 
  Phlip
  http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces