From: mathew <meta@...>
Date: 2006-01-13T01:04:31+09:00
Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings)

*Dean Wampler *<deanwampler gmail.com> writes:
> Let me suggest an XP-style alternative; make thorough unit tests
> required and make sure they "document" - and test! - the design
> "contract".

Unit tests are not an alternative. They are an additional requirement.

>  Some XP gurus, such as Robert Martin, have pointed out
> that the test suite is an executable form of the requirements,
> especially the "acceptance tests" the customer uses (or should use) to
> confirm that a delivery satisfies the requirements.
>   

Although unit tests constitute one set of requirements, they rarely 
cover the full set. In a language like Ruby where code is often generic 
by default, it is implausible to generate enough unit tests to cover 
every eventuality.

For example, consider a simple vector addition routine in a 3D library. 
The unit tests might test its behavior with Float and Integer vectors, 
since that's why it was written. However, there's no reason it shouldn't 
be used with heterogeneous vectors, Complex vectors, or any of a dozen 
other types. Indeed, there's no general way for the writer of the code 
to predict what classes the code might be asked to manipulate, nor 
should they try. (Checking using kind_of? and throwing an exception if 
you don't recognize it is not The Ruby Way, right?)

However, it is of vital importance to document the intended supported 
functionality, which is a superset of the functionality the unit tests 
describe.

Suppose it is determined that the vector add routine is painfully slow. 
An obvious solution is to refactor it to use CBLAS, which provides 
highly optimized native code vector operations. However, to do that you 
need to know whether the feature of supporting (say) Complex vectors or 
BigDecimal vectors is intended or not. The unit tests won't tell you this.

You also need to know whether vectors with elements exceeding the 
capacity of machine types are supported. It's quite likely your unit 
tests don't cover the entire range of integers and floats, because you 
don't have an infinite amount of time to run unit tests in. So, are the 
omissions intentional, indicating a limit to the scope of the API? Or 
are they just omissions? There's no way to tell.

> One limitation of documentation is that it has no enforcement power,
> so you have to write tests anyway to test conformance.

Unit tests have no enforcement power either, because you can just change 
the test. Indeed, I've already had to do this once when it turned out 
that the unit test was wrong. (In net/ftp.)

>  The XP view is
> that you should eliminate the redundancy.

Except it's not redundancy.

Unit tests define a set of functionality that is required. Documentation 
tells you the functionality that is supported, which is generally a 
superset of the functionality required by the unit tests.

People who write code using a library are interested in the *supported* 
functionality, not the bare minimum *required* functionality.

People who want to refactor code need to know the *supported* 
functionality, not the bare minimum *required* functionality.

Finally, a reductio ad absurdum to illustrate the problem in concrete 
form. Give me a function and a set of unit tests, and I can implement a 
hopelessly broken refactored version that happens to pass all those tests:

if (arguments == unit-test-set-1)
  return canned-value-1
if (arguments == unit-test-set-2)
  return canned-value-2
else
  return random-value

Without documentation saying what the function was supposed to do, 
there's no reason my version shouldn't be considered correct according 
to XP dogma. After all, it passes all the unit tests, and it's likely to 
be very fast.


And that's one of the major faults with XP dogma.


mathew