From: "Robert C. Martin" Date: 2001-12-11T13:22:30+09:00 Subject: [ruby-talk:28175] Re: John Roth dolt ( Re: A challenge to proponents of Unit Testing. ) On Mon, 10 Dec 2001 10:23:12 -0800, "Joe Pallas (pallas at cs dot stanford dot edu)" wrote: >Kent Beck wrote: > >An astounding indictment of the way XP approaches Unit Testing. > >> Since I know Sets are insensitive to the order in which elements are added, >> I conclude I don't have to test alternative orders for the arguments. > >So, you have made decisions about what cases to test based on the >implementation. That implies two things: > >1) You have violated the XP rule that tests should be written before the >code that they test. Not quite. Tests, in XP, are written concurrently with the production code. It is quite possible that Kent wrote the first test case, then wrote the code that passed it, and then realized that ordering was independent, and that no test cases needed to be written to test it. > >2) Your test cases are specific to the implementation, and will not >adequately test a different implementation. That's correct. Unit tests are white box tests. They are fragile to significant changes of design. Acceptance tests, on the other hand, are black box tests that are completely insensative to changes in design. You need both kinds of tests. >This means that some future refactoring may make your tests inadequate, >but the coder doing the refactoring would rely on your tests and check >in broken code. One would hope that the programmers who were making the change would review the tests. One would also hope that there was an acceptance test or two that would catch the bug. But in the end, you are right, there are some design changes that might produce bugs that could pass through the tests. I don't consider this a severe problem for most projects since the incidence is not likely to be high. >> What defects am I missing with the above 5 tests? Put another way, what >> further tests could improve the MTBF of my function? > >What a bizarre notion. It is not meaningful to talk about the MTBF of >software, because software does not fail. The term is in use in this thread because someone else used it. >Software is either correct or >incorrect, and that does not change over time. Requirements change, and >execution environments change, but the program itself is not subject to >"bit rot." And still software can have a MTBF. >> If your unit tests are large and complicated, you have a design problem, not >> a testing problem. > >Again, this can only be true if your testing is dictated by the design, >*rather than by the requirements.* That's not just backwards, it is >doubly dangerous in the XP milieu, where frequent changes to the design >are encouraged, and the only validation of those changes comes from the >pre-existing tests. Again, remember that XP has two kinds of tests. White box unit tests and black box acceptance tests. Robert C. Martin | "Uncle Bob" | Software Consultants Object Mentor Inc. | rmartin@objectmentor.com | We'll help you get PO Box 5757 | Tel: (800) 338-6716 | your projects done. 565 Lakeview Pkwy | Fax: (847) 573-1658 | www.objectmentor.com Suite 135 | | www.XProgramming.com Vernon Hills, IL, | Training and Mentoring | www.junit.org 60061 | OO, XP, Java, C++, Python| "One of the great commandments of science is: 'Mistrust arguments from authority.'" -- Carl Sagan