From: Richard Conroy <richard.conroy@...>
Date: 2008-12-10T05:51:32+09:00
Subject: Re: Ruby Test Frameworks

On Tue, Dec 9, 2008 at 3:42 PM, Brian Candler <b.candler@pobox.com> wrote:
> Michael Furmaniuk wrote:
>> Most of the Ruby Test Frameworks that I have
>> looked through are all Web or GUI test frameworks, Test::Unit is nice
>> but I would like to know if there is a Framework for application and
>> system testing that I have missed.  Has anyone used a framework for
>> application and system testing they could recommend, or even an existing
>> one that they have used successfully?
>
> I don't know of anything specific to 'application and system testing'.
> If I were you, I'd start with Test::Unit or its bigger brothers Shoulda
> or Rspec, and see if it does what you actually need.

For the simple stuff, dust is my tool of choice. Its a very simple
wrapper around
Test::Unit, so there is a shallow learning curve, but you get
meaningful test names,
and it is very easy to generate large blocks of tests from arrays of
inputs. Handy
for black box testing, testing regexes etc.

Shoulda and RSpec I think are tackling different needs. The learning
curve is greater
and there is more magic involved, but I suspect that test
organisation, and consolidation
of reporting is probably a greater priority than natural language test
description.

>> Some of my initial requirements are:
>>   - Runs on Windows and Linux
>>   - Supports calling external code/scripts, we have Java and C code that
>> is called for memory utilization and socket testing
>
> You can call external programs using backticks, system() or IO.popen:
> e.g.
>
>  def test_foo
>    result = `foo`
>    assert_equal 0, $?               # exit code
>    assert_match /success/, result   # output on stdout
>  end
>
> You can call actual C using RubyInline or FFI.

In addition, check out JtestR for plain old Java testing (it works great).

>
>>   - Can generate system events for create/modify/delete of files and
>> directories
>
> You can do those actions using FileUtils. Or do you mean you want to be
> *notified* of those sorts of events happening in the filesystem? I'm not
> sure about that.
>
>>   - Has some hooks into Windows Registry, IIS and Linux RPM (Nice to
>> haves but not required)
>
> Not sure, I believe there is Win32 API (but I'm a Linux person).
>
>>   - Can integrate with an IDE (Visual Studio would be nice)

Theres a team integrating Ruby into visual studio, I can't recall offhand.

>
> I think Test::Unit has some graphical runners (but I'm a CLI person :-)

I haven't found a good solution for out of the box test reporting in Ruby.
RSpec has some great reporting, but I need something a bit more turnkey.

I currently use dust to test java code through JtestR on top of an existing
Java build system (so our test reporting is done through JUnit reports and
cobertura works for our code coverage of the java code).