From: Stefan Schmiedl <s@...>
Date: 2001-12-03T06:59:08+09:00
Subject: [ruby-talk:27294] Re: Quantitative Unit-testing? (Re: John Roth dolt (  Re: A challenge to proponents of Unit Testing. )

unit tests in isolation are as much evil as every other part
of a software building process in isolation.

Panu Viljamaa (2001-12-03 06:00):

> 1.  If we start system building from tests first,  we could
> declare that the tests *are*  the "Specification" of the system.
> Then the question turns into: "How can we know all assumptions
> other components make about my component are expressed by my
> unit tests?".  I.e. how can we know nobody is violating the
> contract/specification expressed by my tests-code?

if someone does, you will receive a bug report. you can add
another test thus enhancing the interface or tell the other to
rtfm and leave things as they are. without feedback things won't
work.

> 2.  If on the other hand unit tests are *not* taken as the
> 'specification' of a component,  we should ask: "How can we know
> the existing unit tests prove all specified requirements are
> being fulfilled?"  And if  we can't prove this, how can we have
> any qualitative assurance about  the 'correctness' or even
> 'quality' of our code ?

you have functional/acceptance/user tests controlling the
behaviour on a larger scale. and why do you need to "prove"
something? the ultimate criterium of usefulness is whether
something works and if it does not, fails gracefully.  there is a
well-known saying of don knuth regarding this matter.

> 
> 3.  I believe (automated) unit tests are a good technique for
> discovering bugs due to changes made to the system, early on.
> But it is misleading to imply that since you have written and
> run *some* tests, your system now works "correctly". Yet this is
> what  a consultant might tell a customer: "Look, we have all
> these unit tests in place, and we run them all without errors!
> We don't need interface contracts. We don't need specifications.
> Our code is simply the best since it passes all unit tests we
> have written!"

shoot the consultant, if you meet him next time. ;>

this is the same consultant that sells you other methodologies
with other inherent flaws by promising other stuff that cannot be
guaranteed.  it's a fault of the consultant, not of the unit test.

unit tests help you immensely in diagnosing problems, both early
and late in development. they are no silver bullet. you still need
to apply whatever knowledge you have acquired.

but do you believe that you are completely healthy, only because a
doctor told you so? 

> 
> -Panu Viljamaa
> P.S.
> Wouldn't it intuitively make sense to have another set of people
> writing the tests, than the ones writing the code which must
> pass those tests?

do it and report problems and successes to the rest of us, please.
meanwhile you might want to break your lines around colum 70 to
improve readability of your posts.

s.
-- 
Stefan Schmiedl
EDV-Beratung, Programmierung, Schulung
Loreleystr. 5, 94315 Straubing, Germany
Tel. (0 94 21) 74 01 06
Public Key: http://xss.de/stefan.public

shhhh ... I can't hear my code!