From: Warren Brown <wkb@...>
Date: 2003-01-28T09:04:03+09:00
Subject: Re: Test::Unit -> order of tests?

> To implement this, I would suggest renaming all your test_XXX methods
> to subtest_XXX and have a single test_all method that calls the
> subtest_XXX methods in the proper order.  Have your setup method
> create the database and your teardown method delete it.

    This is a *great* solution for test cases that must be run in a certain
order.

    The problem that I run into is when I have some test cases that test
simple building-block methods, and another big test case that tests a method
that calls those building-block methods.  If a bug pops up in one of the
building-block methods, it would be nice to have the building-block's test
case catch it rather than the big test case, since the big test case may
fail in an obscure way if the building-blocks it depends on are
untrustworthy.  The tests *can* be run in any order, it's just more helpful
to have them run in the given order.

    My suggestion is to have the test cases execute in the order they are
encountered, but add a 'randomize' capability that can be specified from the
command line to run the test cases in random order.  I can see no benefit in
having the test cases execute in alphabetical order, but if there is a
benefit, that too could be a command line option.

    - Warren Brown