From: Bill Kelly <billk@...>
Date: 2004-06-07T10:38:41+09:00
Subject: Re: How to ducktype a Hash?

Hi,

From: "Sean O'Dell" <sean@celsoft.com>

> > I have only a small amount of experience with using Ruby on large
> > projects, so maybe some others can chime in here. You said earlier that
> > you were not using unit tests any more because they seemed like too much
> > work, but they did not save any time in the debugging game (forgive me
> > if I have misquoted you). I wonder if what you are finding is that Ruby
> > w/o Unit Tests is much harder to use on large projects than Ruby w/ Unit
> > Tests.
> 
> No.  As the project grows, errors crop up in new places.  Unit tests only help 
> if either a) you can anticipate where the errors are going to occur or b) 
> errors keep coming back to the same points in code where you've put unit 
> tests to debug the bug originally.

[...]

> Type checking and unit testing are not the same thing at all.  Unit testing is 
> a develop-time only event, and even then you do it sporadically.  Type 
> checking is done at run-time.
> 
> I have this thing where I automatically start to de-value a person's opinion 
> when I detect "unit test evangelisation."  I've done a lot of unit tests, and 
> I've written my own framework, and I've used them on large and small 
> projects, and they eat up time and don't help me catch any errors I couldn't 
> already anticipate.  Whenever someone tells me unit tests help them a lot, I 
> have come to assume that anticipating bugs must be inordinately difficult for 
> them.  I simply don't have that sort of trouble.

There seems to be a contradiction here.  You're reporting a
non-zero bug rate (tests didn't help, "As the project grows,
errors crop up in new places.")  But when people tell you they
are experiencing bug rates so low that their defect-tracking
software starts gathering tumbleweeds and is eventually 
decommissioned for lack of relevance...  Well I guess just
saying that unit tests "help them a lot" doesn't quite have
the same impact.  But I've seen reports from what I recall
as numerous developers saying exactly this for at least
medium-sized projects in both dynamic and static typed 
languages.  (In my case one module on a project was an
HTML Parse / Render / Layout engine in JPython in a mostly
Java-based consumer app.  The ability to do duck typing
was one of the *reasons* I preferred Python to Java.  I do
not recall any significant bugs in that code, let alone ones
attributable to duck typing goofs.)

Or maybe you're like me and can write software with a low
defect rate without automated testing.  I used to do a 
bang up job with just ASSERT() macros in C.

But what I get out of code developed test-first is extreme
flexibility.  Test-first development is hard for me, I
feel like I'm going slower in the short term, all the time.
But I notice the my flexibility to tear up the program and
reconfigure it to develop new features requested by the
customer incrementally, and redesign old parts of the program
when they are no longer adequate to accommodate new features,
is significantly improved by having a unit-test network
supporting my refactoring.  It also, I find, boosts my 
confidence in releasing frequently to users, or to a live
server environment, while development is underway of new
features, without having some lengthy "code lockdown" testing
phase bug fixing phase before putting out a new release.  Which
is important to me.

So my coding goes slower, but I can add continual refactoring
into the mix (which I couldn't do prior to test-first development).
And I feel continual refactoring is a big enough benefit to
make test-first development worth it.

> Unit testing is a develop-time only event, and even then you do
> it sporadically.

Correct me if I'm wrong, but it sounds like you've tried 
Unit testing, but not test-first development.  Sporadically?

> On large projects, way before unit testing reared it's fanatical head

I've done the write low-defect code without unit tests
approach for many years.  But I find with a unit testing
network I can do aggressive continual refactoring, which
I think is of sufficient benefit to warrant the difficulty
imposed by unit testing (like I said, it's hard for me.)

So that leads me to wonder if refactoring is less of an
issue for you, or unit tests didn't help you refactor
more easily?  Or whatever, something is making your
experience with unit tests different from mine.

But - if so I'm interested.  I hope that our experiences
being different doesn't automatically imply fanaticism
on my part.


Regards,

Bill