From: Phrogz <gavin@...>
Date: 2007-03-07T06:30:06+09:00
Subject: Automatic Benchmark Iterations

When performing a benchmark comparison of several techniques, this
idiom is very common for me:

  N = 1_000_000
  Benchmark.bmbm{ |x|
    x.report( 'foo' ){
      N.times{
        # foo code
      }
    }
    x.report( 'bar' ){
      N.times{
        # bar code
      }
    }
    x.report( 'jim' ){
      N.times{
        # jim code
      }
    }
  }

Observation #1: It's annoying to have to type "N.times{ }" for each
report. I also want the exact same value, and I always want some
iterations.

Consequent Desire #1: It'd be nice if I could write the above as:
  Benchmark.bmbm( 1_000_000 ){ |x|
    x.report( 'foo' ){
        # foo code
    }
    x.report( 'bar' ){
        # bar code
    }
    x.report( 'jim' ){
        # jim code
    }
  }

Observation #2: I'll frequently start with a 'guess' value of
N=1_000_000, but if it's taking too long, I'll start playing with
values of N until I get something that takes 1-5 seconds for the first
compared value.

Consequent Desire #2: It'd be cool if I could write the above as:
  Benchmark.autobm{ |x|
    x.report( 'foo' ){
        # foo code
    }
    x.report( 'bar' ){
        # bar code
    }
    x.report( 'jim' ){
        # jim code
    }
  }
...and it would figure out an appropriate number of iterations for me.
(One approach would be to time one iteration, figure out an
appropriate multiple, try benchmarking that number of iterations, and
adjust the number iterations if the result is outside some desired
range. Rinse/repeat.)


The Benchmark::Report#item method (aka #report) is implemented as
this:

    def item(label = "", *fmt, &blk) # :yield:
      print label.ljust(@width)
      res = Benchmark::measure(&blk)
      print res.format(@fmtstr, *fmt)
      res
    end

My question: What is the least-overhead way you can come up with to
wrap the supplied block in an iteration-controlled loop? For example:

  def item( label="", *fmt, &blk )
    ...
    res = Benchmark::measure( &lambda{ for i in 1..@iterations;
blk.call; end } )
    ...
  end