From: mathew Date: 2006-01-14T02:26:16+09:00 Subject: Re: Design contracts and refactoring (was Re: mathn: ugly warnings) Dean Wampler writes: > The redundancy part (or DRY in Ruby parlance) I > was referring to is the goal of only writing the requirements once and > it's better to write them in an executable format so they have > enforcement power. But to go back to the problems I was pointing out: a) In the real world, the requirements do not constitute the entirety of the supported functionality. b) In the real world, actually unit testing the complete scope of the requirements requires unreasonable amounts of time and effort. Consider '+'. Requirements are that it add any two things which constitute numbers. Trouble is, you can't even test every possible pair of floating point numbers, let alone the integers and rationals. Any test of N pairs that you do will be statistically insignificant. > So, the acceptance tests used by customers to confirm that > requirements are met may be written in a documentation tool from which > test code is generated. Right. Unit tests are not the requirements; they are acceptance tests used to confirm that the requirements have been met. There's a subtle but very important difference. Similarly, requirements are not specifications. The specifications are (hopefully) derived from the requirements, but they typically describe a superset of the functionality strictly required, and almost always describe a superset of the functionality tested. For example: Requirements: method to add two vectors of Floats. Specification (as implemented): method adds two vectors of objects which support '+'. Unit tests: take a sample file of vectors of floats, add them in pairs, compare against the file of correct output. All perfectly reasonable. However, the unit tests do not tell you whether anything other than floats will work with the method. That's perfectly reasonable, because working with something other than floats wasn't a requirement. However, it's also perfectly reasonable that the first implementation happens to work with integers, rationals, and so on--because the power of Ruby is that generic programming doesn't take extra effort. In fact, it would take extra effort to prevent the code from working with other types. So, if you only provide unit tests and no documentation, users have to guess. They might look at the unit tests, and guess that the method only works with floats. End result: code bloat because they go and unnecessarily find or implement an integer vector add. Or, they might look at the code, and see that it works with Complex. Then they go and write a ton of scientific visualization software that relies on the fact. Then you want to refactor the method to use CBLAS for speed, and it breaks NASA, and there are howls of anguish. Hopefully I'm getting the problem across better this time around. I am *not* discussing unit tests vs documentation, or unit tests vs contracts, or unit tests vs anything else, or documentation vs anything else. At no point have I said that unit tests are anything other than essential. My point is simply that a documented API is *also* essential, and at a minimum it should describe the functionality that is *supported*, as opposed to that which is tested or that which happens to work. Leaving people to guess the supported API from the unit tests (or worse, the code) is a flawed approach which has led to problems refactoring. In a nutshell: documentation = what is *supported* or *supposed to work* unit tests = what is *actually tested* code = what *happens to work* What is *tested* != what is *supported*, because you can't test everything. I can't feasibly test every possible RSS feed against my RSS parser, for example. What is *supported* != what *happens to work*, not least because Ruby code tends to be generic by default. Even ignoring the generic issue, though, just because my RSS parser accepts a particular malformed feed today, doesn't mean I'm necessarily going support that malformed feed after I refactor the parser. And when I'm using libraries, though I do also care about what's tested, my primary concern is what's supported. I care about knowing that something I use today won't change in an unexpected way and break my application. e.g. Rake, Yaml, ... mathew [ And as usual, as soon as I get into a discussion, the mailing list software stops sending me mail. ]