From: nathaniel@...
Date: 2003-01-09T04:56:44+09:00
Subject: Re: Test::Unit fails w/no tests [was: testunit 0.1.6 problems]

Gavin Sinclair [mailto:gsinclair@soyabean.com.au] wrote:

> A failure, in a test unit sense, means strictly (to me) that 
> an assertion has failed, and an error means strictly that an 
> uncaught exception was detected.

In general I agree, although I'm trying to figure out how important the
'strictly' part is. I guess you could think of this failure as just the
default test in TestCase; if there are no other tests, a default test is
run that always fails.


> I see no need to mark the lack of tests with a failure: one. 
> it's misleading;

Why is it misleading? The failure message seems fairly clear (although
clarity is one thing there's always room for more of):

  Loaded suite TC
  Started

  Failure!!!
  run:
  No tests were run.

  Finished in 0.0 seconds.
  0 tests, 0 assertions, 1 failures, 0 errors

The one thing about it that I think IS misleading is that, if you bundle
up an empty TestCase with a non-empty TestCase and run them together,
you won't get a failure anymore - the check is only made at the runner
level. I need to at the very least make that consistent.


> and two, it's inconvenient. I haven't done 
> it myself, but I would do this: decide my tests for a certain 
> class are no longer valid, delete them, but leave the shell 
> (the TestCase) there so I can create some new ones in the future.

I both agree and disagree with the inconvenience of it. Yes, if you keep
around a lot of unused TestCases, it would be somewhat inconvenient, but
I'm not sure that's a practice that ought to be supported. As an XP'er,
I'd say, "YAGNI!" and delete the TestCase as soon as it's empty, knowing
that it's cinchy to add back. If I know that I'm going to add another
test to that TestCase in a minute, then the failure is a boon, because
it keeps me from getting distracted and forgetting to add the test.

I guess what I'm saying is that the failure wouldn't hurt my way of
working, and might even help it sometimes. It's not my goal, however, to
impose my development methodology on others, so it could very well be
that this ought to go.


> I suggest emitting a warning in hte event of an empty 
> TestCase.

Warnings in tests are too mushy for me; I want red or green, true or
false. I'm much more inclined to drop the failure altogether than to add
the concept of warnings to the framework.


I'm leaning towards leaving the failure, but making it more consistent
as noted above. My mind isn't really made up yet, though, so I'd like to
hear some more on this first (if there are any other opinions). I'm also
going to try to dig up the original request for this feature to present
"the other side", as it were.

Thanks for listenin'


Nathaniel

<:((><
+ - -
| RoleModel Software, Inc.
| EQUIP VI