From: Phlip <phlipcpp@...>
Date: 2007-10-01T07:20:06+09:00
Subject: Re: Optimizing testing

Martin Martinos wrote:

> I have a rake task that lunches all my unit tests. The execution of
> those tests may take over 20 mins.  I would like to have two rake task,
> one that executes a set of quick tests and one that executes all the
> tests that will be executed in a nightly build.

> For the moment I create a test suite that run all the TestCase methods
> that its name finishes by _long

To run only the long ones, I'd do this:

rake test TESTOPTS=-n/_long$/

Note that's selects for long cases, and we need another Regexp that only 
selects fast tests. One bug in all Regexps is they are infernally hard to 
negate. Regexps rely on their host language to call !~, instead of simply 
providing a negator. So you could put together some negator for _long$ from 
primitives, but I suspect you can't just call something simple, like 
!(_long$). I hope to be corrected here!

Next, the point of test _suites_ (the things that inherit < 
Test::Unit::TestCase) is to categorize your cases by their test situations. 
_Not_ necessarily to categorize them by their targets. You should put all 
the long tests into a big suite, SlowSuite, and then flag it like this:

  if ENV.has_key? 'slow'
    class SlowSuite < Test::Unit::TestCase
      ...
    end
  end

Then you just call 'slow=true rake test', or 'rake slow=true test', to also 
get the slow ones.

Next, long tests are a symptom of insufficient decoupling. If your tests, in 
TDD terms, are helping force your dependencies apart, then new tests should 
usually be narrower and faster than old ones. Not flabbier and slower. I get 
back to you when I fully obey this advice myself!

-- 
  Phlip
  http://www.oreilly.com/catalog/9780596510657/
  "Test Driven Ajax (on Rails)"
  assert_xpath, assert_javascript, & assert_ajax