[ruby-core:17753] Re: Release Plan: Ruby 1.9.0-2

Hi,

I stop to object that Test::Unit is replaced with miniunit.
Because nobody objects it except me. It will mean that my
opinion doesn't make sense. If Matz says 'go', Test::Unit
will be replaced with miniunit.

I'll write my opinions in this mail below. I'm happy if my
opinions are considered a bit in the future. It's OK for me
if they aren't considered.


In <[e-mail]>
  "[ruby-core:17389] Re: Release Plan: Ruby 1.9.0-2" on Tue, 24 Jun 2008 10:21:45 +0900,
  Ryan Davis <[e-mail]> wrote:

> >  * miniunit isn't extensible.
> 
> false. it is ruby. it is just as extendable as test/unit, if not more,  
> since it is cleaner.

It seems that test/unit isn't easy to extend. So miniunit
isn't easy to extend. An example is below.

> > === miniunit isn't extensible
> >
> > Unfortunately Test::Unit doesn't keep improving itself for a
> > few years. But there are many improvements in the
> > world. e.g.: RSpec provides BDD style syntax, multiple
> > setup/teardown (before/after) mechanism, new test (example)
> > status (pending) and so on. RSpec and Mocha provides a mock
> > system.
> >
> > Some advanced users (like ActiveSupport, Mocha and so on)
> > extend Test::Unit by overriding existing methods and/or with
> > Test::Unit's internal API because Test::Unit doesn't provide
> > extensible interface. It causes ugly hacks.
> 
> miniunit is compatible with ActiveSupport already. Chad Fowler did two  
> 1-line changes to make miniunit work with flexmock. I'm sure mocha  
> will take a change or two, but it won't be much.
> 
> The ugly hacks are required for test/unit, not for miniunit. Making  
> miniunit work with these hacks has been ugly and painful, but it is  
> something we can address with the library authors and clean up over  
> time.

e.g. active_support/testing/setup_and_teardown.rb:

For test/unit:

  def run_with_callbacks_and_testunit(result) #:nodoc:
    return if @method_name.to_s == "default_test"

    yield(Test::Unit::TestCase::STARTED, name)
    @_result = result
    begin
      run_callbacks :setup
      setup
      __send__(@method_name)
    rescue Test::Unit::AssertionFailedError => e
      add_failure(e.message, e.backtrace)
    rescue *PASSTHROUGH_EXCEPTIONS
      raise
    rescue Exception
      add_error($!)
    ensure
      begin
        teardown
        run_callbacks :teardown, :enumerator => :reverse_each
      rescue Test::Unit::AssertionFailedError => e
        add_failure(e.message, e.backtrace)
      rescue *PASSTHROUGH_EXCEPTIONS
        raise
      rescue Exception
        add_error($!)
      end
    end
    result.add_run
    yield(Test::Unit::TestCase::FINISHED, name)
  end
  alias_method :run, :run_with_callbacks_and_testunit

For miniunit:

  def run_with_callbacks_and_miniunit(runner)
    result = '.'
    begin
      run_callbacks :setup
      result = super
    rescue Exception => e
      result = runner.puke(self.class, self.name, e)
    ensure
      begin
        teardown
        run_callbacks :teardown, :enumerator => :reverse_each
      rescue Exception => e
        result = runner.puke(self.class, self.name, e)
      end
    end
    result
  end
  alias_method :run, :run_with_callbacks_and_miniunit

"ugly hacks" as I said means that overriding existing method
with using internal API and aliasing it. Example internal
API is add_failure for test/unit case, runner.puke for
miniunit.

If test/unit and/or miniunit are/is easy to extend,
active_support/testing/setup_and_teardown.rb doesn't need to
overriding existing run method.

> > miniunit doesn't provide extensible interface because it
> > introduces complex mechanism. But we need it to avoid
> > ugly hacks.(*) If miniunit keep simple, we will be dirty.
> 
> I disagree. I've found a number of people (besides just me) that find  
> extending miniunit to be MUCH MUCH easier than test/unit. I rely on  
> basic idiomatic ruby to make miniunit much more approachable. See Phil  
> Hagelberg's previous email as evidence.
> 
> Here is a real world (idiomatic) example of test/unit extension vs  
> miniunit:
> 
>    def assert_sorted(actual, message=nil, &block)
>      expected = actual.sort(&block)
>      message ||= "Expected order:\n#{expected.inspect}\nbut got order: 
> \n#{actual.inspect}\n"
>      assert_block(message) { expected == actual }
>    end
> 
>    vs:
> 
>    def assert_sorted(actual, message=nil, &block)
>      expected = actual.sort(&block)
>      assert_equal expected, actual, "Order is wrong:"
>    end

I couldn't understand why test/unit version is:

>    def assert_sorted(actual, message=nil, &block)
>      expected = actual.sort(&block)
>      message ||= "Expected order:\n#{expected.inspect}\nbut got order:\n#{actual.inspect}\n"
>      assert_block(message) { expected == actual }
>    end

We can write it same as miniunit version:

>    def assert_sorted(actual, message=nil, &block)
>      expected = actual.sort(&block)
>      assert_equal expected, actual, "Order is wrong:"
>    end


> > I want users to be simple rather than testing framework is
> > simple.
> 
> I want both to be simple. These aren't mutually exclusive.

I don't think so in many cases. It may be true in some cases.


> > (*) How do we add new command line option? How do we get
> >    colorized output? How do we get diff between expected
> >    and actual values? Need another filter command? Need to
> >    overriding existing methods? It doesn't conflict with
> >    other extension?
> 
> And you think that these are addressed better in test/unit? Tell me...  
> where in the files below does colorized output go? How about  
> commandline options? Filtering?
> 
> % find . -name \*.rb | xargs grep -l Filter
> ./unit/assertions.rb
> ./unit/error.rb
> ./unit/testcase.rb
> ./unit/util/backtracefilter.rb
> % find lib/test -name \*.rb | xargs wc -l
>        14 lib/test/unit/assertionfailederror.rb
>       622 lib/test/unit/assertions.rb            # HERE?
>       220 lib/test/unit/autorunner.rb
>       107 lib/test/unit/collector/dir.rb
>        34 lib/test/unit/collector/objectspace.rb
>        43 lib/test/unit/collector.rb
>        56 lib/test/unit/error.rb                 # HERE?
>        51 lib/test/unit/failure.rb
>       160 lib/test/unit/testcase.rb              # HERE?
>        80 lib/test/unit/testresult.rb
>        76 lib/test/unit/testsuite.rb
>       127 lib/test/unit/ui/console/testrunner.rb
>       268 lib/test/unit/ui/fox/testrunner.rb
>       416 lib/test/unit/ui/gtk/testrunner.rb
>       465 lib/test/unit/ui/gtk2/testrunner.rb
>        68 lib/test/unit/ui/testrunnermediator.rb
>        46 lib/test/unit/ui/testrunnerutilities.rb
>       260 lib/test/unit/ui/tk/testrunner.rb
>        40 lib/test/unit/util/backtracefilter.rb # HERE?
>        90 lib/test/unit/util/observable.rb
>        48 lib/test/unit/util/procwrapper.rb
>       280 lib/test/unit.rb
>      3571 total
> 
> % find lib/mini -name \*.rb | xargs wc -l
>        31 lib/mini/mock.rb
>        82 lib/mini/spec.rb
>       436 lib/mini/test.rb                      # ALL HERE
>       549 total

colorized output:
        127 lib/test/unit/ui/console/testrunner.rb # ALL HERE

commandline options:
        220 lib/test/unit/autorunner.rb # ALL HERE

IMHO, putting all features in a large file isn't related to
easy to extend.

> > === miniunit just provides minimal features
> >
> > miniunit provides some advanced features: mock system and
> > BDD style syntax. But they just provides limited
> > functions. e.g. Mini::Mock can't handle multi expects for
> > the same name.
> 
> I don't really see how mini/spec is limited. I talked to David  
> Chelimsky at railsconf only feature it is really lacking from rspec is  
> nested contexts.

I can't find a feature that is corresponding should_receive.
IMHO, powerful mock support is the very useful feature in
RSpec.

And it seems that defining custom must_* method is a
bother. Should we use Object.infect_with_assertions?

> > Most of users prefer to useful system rather than simple
> > system but limited features because they want to write their
> > tests simply like Ryan wants miniunit to be simple.
> 
> again... I'm not seeing this as a problem. miniunit is simple. it also  
> supports nearly every project out there already. For those that  
> require test/unit internals, things are actually cleaner now. They can  
> either require test/unit as a gem, or miniunit... it'll still work on  
> 1.8 and 1.9.
> 
> > Most of users will use RSpec rather than Mini::Spec if they
> > want to use BDD style syntax.
> 
> yup. how is this a problem? This is only 82 lines of overhead.
> 
> > Most of users will use Mocha or RSpec rather than Mini::Mock
> > if they want to use a mock system.
> 
> yup. how is this a problem? This is only 31 lines of overhead.

I want said that: Their only has limited features. So their
are not miniunit's benefits. They are needless
features in the standard testing framework. They are just
simple.

> > Yes, I think that miniunit is very good solution for Ruby
> > implementation developers because it has very small
> > dependencies. It doesn't use standard libraries like
> > OptionParser. But users doesn't require very small
> > dependencies. They want useful tools rather than very small
> > dependencies.
> 
> again... I don't see small as bad.

I didn't say that small is bad. I just said that too minimal
is bad for users convenience.


Thanks,
--
kou