From: Bill Kelly <billk@...>
Date: 2002-08-10T09:43:42+09:00
Subject: Re: Unit testing is considered harmful


From: "MikkelFJ" <mikkelfj-anti-spam@bigfoot.com>
>
> I'm sorry, I could help it. I just find this considered harmful thing
> completely overused.
> Anyway todays topic is about that narrowminded focus on Unit Tests.
> 
> We just saw the programming contest having the test program tested, rather
> than the applications. Of course UnitTest never claimed to be complete.

Coincidentally, I submitted a question about this--CodingChallenge0002
vs. unit tests--to the extreme-programming mailing list this morning.
I was commenting on my surprise that we kept adding unit tests after
the fact, and breaking existing solutions.  I referred people to the
wiki page.  What happened is people replied back with detailed 
questions, specifically stating that the requirements were too vague
and asking for clarification. . . . Which is just what happened here on
ruby-talk when the challenge was first announced.  [[Not pointing 
fingers!! I enjoyed the challenge and learned a lot from it.]]

[...]
> If we take programming contest as example, it is important to identify the
> nature of the problem. I suggested a relation to the assingment problem,
> which may or may not be appropriate. But that view gives a good idea of
> which data sets are valid and which a not, instead of extrapolating from a
> few unit test cases.

I agree.  In no way when I approached the problem, was I extrapolating
from what unit test cases were present.  I had asked David on the list
for clarifications to a couple questions regarding duplicates and whether
the items could be considered to be sortable.  I also read David's reply
to someone else where he described thinking about the problem as drawing
lines, connecting elements in the 1st array to rows in the second. . . 
My point is two-fold: one, that one or more of us should probably have
considered going back to the wiki page and updating the requirements to
include the new information/clarifications expressed on ruby-talk; and
two, that I--at the time--had no problem adding further unit tests as
we went along, thinking of it as codifying what was already clear to
me in the requirements (forgetting, that the requirements were in
existence now partially on the wiki page, but partially in ruby-talk
messages...)

So I was somewhat at a loss as to how new unit tests were breaking so
many of the existing solutions...  But I think, as was brought to my
attention on the XP list, that the essence of the problem may have 
lied in the incomplete definition of the requirements on the wiki page,
and . . . . well.  Pretty much that. :)

However, I don't, myself, see how this could be construed as a failure
of the unit tests...  Unless it was thought that the unit test cases
were to substitute as implicit definitions of the requirements.  I 
would agree that we may have witnessed *that* approach being harmful.
But I still don't blame the unit tests themselves........


Just my sqrt(4) cents...


Regards,

Bill