From: Benoit Daloze <eregontp@...>
Date: 2011-02-09T18:12:43+09:00
Subject: Re: Benchmark in MiniTest

On 7 February 2011 01:15, Oren <orengolan@gmail.com> wrote:
> https://gist.github.com/813736
>
>  def test_hot_is_constant
>    assert_performance_constant 0.9999 do
>      @reddit.hot
>    end
>  end
>
>  def bench_my_algorithm
>    assert_performance_linear 0.9999 do |n| # n is a range value
>      n.times do
>        @reddit.hot
>      end
>    end
>  end
>
> the output i get is:
> TestReddit      1       10      100     1000    10000
> bench_hot        0.000039        0.000052        0.000032        0.000033        0.000032
> bench_hot2       0.000046        0.000047        0.000088        0.000596        0.005650
>
>
> my question - Is there a way to set the maximum time for a method to run?
> if it goes above it i want it to fail.
> what does assert_performance_constant mean? is it a constant speed in relation to the argument the method accepts?
>
> Thanks
>
>

Please be aware your are testing if Integer#times is liner in your
second test, not @reddit.hot.
(which does not take any parameter from your code, and is then very
likely to have constant time).

A good example would be:

def fib(n)
  # my super code to compute the n^th Fibonacci term in linear time
end

assert_performance_linear 0.9999 do |n|
  fib(n)
end

The only good usage of Integer#times (except for testing #times
itself) in assert_performance_*, is to increase the time when the
block runs too fast (like all n <= 10 000 runs in less than 0.001 s)
You then want to use a constant to call #times on it (like 1000.times
{ fib(n) }), *never* n.
You could also use the :bench_range option, probably better.

Regards,
Benoit Daloze