From: Daniel Schierbeck <daniel.schierbeck@...>
Date: 2007-03-08T01:54:05+09:00
Subject: Re: A better benchmarking syntax (was: Automatic Benchmark	Iterations)

On Wed, 2007-03-07 at 23:18 +0900, Trans wrote:
> 
> On Mar 7, 8:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
> wrote:
> > Reading Phrogz' post about automatic benchmark iterations, and then
> > seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> > might need a friendlier syntax for benchmarks all together. Minutes
> > later, I discovered that someone had almost the same idea as me[2].
> > Anyway, I'd like to just throw it out here, and hear what you people
> > think.
> >
> > The idea is to make benchmarks syntactically similar to the current
> > Test::Unit test cases, e.g.
> >
> >   class SortBenchmark < Benchmark
> >     def setup
> >       @array = [1, 6, 2, 9, 4, 6, 2]
> >     end
> >
> >     def teardown
> >       @array = nil
> >     end
> >
> >     def report_quicksort
> >       @array.quicksort
> >     end
> >
> >     def report_mergesort
> >       @array.mergesort
> >     end
> >   end
> >
> > Automatic iteration could be added, either as a class method call or by
> > creating a different base class (IterativeBenchmark?)
> >
> > So, what do y'all think? If I'm not the only one liking this, I might
> > whip something up when I get some spare time...
> 
> Cool! Go BDD with it:
> 
>   benchmark "compare sorting methods" do
> 
>     compare "quick sort"
>       @array.quicksort
>     end
> 
>     compare "merge sort" do
>       @array.mergesort
>     end
> 
>   end
> 
> Or something like that.

I must admit to being a great fan of RSpec, although the pace of API
changes has thrown me off for now, but I'm not sure this path is the
best. I did actually think about it at first. The strength of Test::Unit
is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
like to add the same simplicity to any new benchmarking system. I'll
give it some more thought :)

Any idea for a name? "benchmark" is unfortunately already taken...


Cheers,
Daniel