From: Ryan Davis <ryand-ruby@...>
Date: 2011-02-08T08:41:26+09:00
Subject: Re: Benchmark in MiniTest


On Feb 6, 2011, at 16:15 , Oren wrote:

> https://gist.github.com/813736
> 
>  def test_hot_is_constant
>    assert_performance_constant 0.9999 do
>      @reddit.hot
>    end
>  end
> 
>  def bench_my_algorithm
>    assert_performance_linear 0.9999 do |n| # n is a range value
>      n.times do
>        @reddit.hot
>      end
>    end
>  end
> 
> the output i get is:
> TestReddit      1       10      100     1000    10000
> bench_hot        0.000039        0.000052        0.000032        0.000033        0.000032
> bench_hot2       0.000046        0.000047        0.000088        0.000596        0.005650
> 
> 
> my question - Is there a way to set the maximum time for a method to run?
> if it goes above it i want it to fail.

No. I'm not sure it makes too much sense. On a slower or more loaded machine, you'd get false negatives.

> what does assert_performance_constant mean? is it a constant speed in relation to the argument the method accepts?

8511 % ri MiniTest::Unit::TestCase.assert_performance_constant
= MiniTest::Unit::TestCase.assert_performance_constant

(from gem minitest-2.0.2)
------------------------------------------------------------------------------
  assert_performance_constant(threshold = 0.99, &work)

------------------------------------------------------------------------------

Runs the given work and asserts that the times gathered fit to match a
constant rate (eg, linear slope == 0) within a given error threshold.

Fit is calculated by #fit_constant.

Ranges are specified by ::bench_range.

Eg:

  def bench_algorithm
    assert_performance_constant 0.9999 do |x|
      @obj.algorithm
    end
  end