From: David Mitchell <monch1962@...>
Date: 2008-07-15T21:39:19+09:00
Subject: Re: Is it possible to dynamically extend Test::Unit test cases?

Thanks Phlip,

The reason I need to do this is that we've got a small Watir-based DSL
written to allow us to drive an app through code that looks sort of
like:
login("fred", "password")
click_tab("Reports")
click_drilldown("Asia")
open_report("Some report title")
...

It essentially lets us construct test cases in something like plain English.

We built a few test cases using the DSL with a "normal"
Test::Unit::TestCase approach, then showed them to our testers.
Everyone was pretty excited about it; we can generate our own test
data using the DSL, the testers can comprehend the DSL without having
to dig into the nuts and bolts of the application itself, we can
finally build a full regression test suite for an application that's
basically a pig to drive using normal automation testing tools like
QTP, the scripts we write using the DSL are easy to maintain over
time, and everyone's happy.

Once our testers got a look at that, they pointed out what should've
been obvious all along: we can now get actual business users to write
a lot of the test cases using the DSL, rather than using specialist
testers.  Rather than writing huge business requirements documents
that have a habit of getting misinterpreted, we can get the business
users to create what are essentially test cases using the DSL, and
that gives the developers a reasonably unambiguous description of how
things are supposed to work - we'll save a whole lot of time and money
we're currently wasting translating between business-speak and
developer-speak.

The only problem was the "scaffolding" code; apparently business users
are incapable of writing/extending code that looks like

class Testcases < Test::Unit::TestCase
  def test_1
    <<DSL stuff here>>
  end
  def test_2
    <<more DSL stuff here>>
  end
  ...
end

but they are capable of creating a bunch of test cases in individual
files that contain nothing but the DSL commands.  They'll use e.g.
Notepad to create test cases in individual files that look like:
login('fred', 'password')
click_tab('Reports')
...

Fine with me - I just work here...

So now I've got a situation where we're going to have business users
generating loads of test cases using our DSL (without any of that
nasty complicated Test::Unit::TestCase stuff), saving them in flat
files, and we need to run be able to run some unspecified number of
test cases that will change over time.  What I need to be able to do
is something like:

Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
  <<grab the content of each file, build a new Test::Unit::TestCase
wrapper around it and eval it>>
end

That's no problem - I've got most of this working already; all I
needed was the way to dynamically add new test cases, and you've now
given me a way to do that.  I'll have a play with it tomorrow.

Thanks again

David Mitchell

2008/7/15 phlip <phlip2005@gmail.com>:
> David Mitchell wrote:
>
>> The test cases run as expected, but now I need to add some new test
>> cases at run time.
>
> Why? Just curious...
>
>> What I think I need to do is something like this
>
> Try this:
>
> class MySuite < Test::Unit::TestCase
>
>  [:concept_1, :concept_2, :concept_3].each do |thang|
>    define_method "test_#{thang}" do
>      assert_concept thang
>    end
>  end
>
>  def assert_concept(thang)
>    #  now convert the thang symbol to a real thing and test it
>  end
>
> end
>
>> t.send(:define_method, "test_defined_at_run_time", "")
>
> You are trying too hard. define_method is easier than that to call.
>
> And note I put most of the processing _outside_ the define_method. Its only
> job is to construct a test case name and pass in a thang. This helps make
> assert_concept() more comprehensible.
>
> --
>  Phlip
>
>