From: "Berger, Daniel" <Daniel.Berger@...>
Date: 2005-04-06T01:42:42+09:00
Subject: Re: Benchmarking whole programs

> -----Original Message-----
> From: gabriele renzi [mailto:surrender_it@remove-yahoo.it] 
> Sent: Tuesday, April 05, 2005 10:35 AM
> To: ruby-talk ML
> Subject: Benchmarking whole programs
> 
> 
> Hi gurus and nubys,
> 
> I was playing with the  ruby programs in the alioth language 
> shootout and I realized I have no idea on how to benchmark 
> them all. What I mean is: If I want to benchmark a single 
> function I can use the 
> usual benchmark code also shown in ri, but what if I want to 
> check two 
> slightly different implementations of a a single program?
> 
> I can't just stick them all in a #meth1 and #meth2 thing, since there 
> may be constants defined (actually they almost surely will, at least 
> classes and modules), I suppose I could do something like:
>         Benchmark.bm do |x|
>           x.report {require 'first_impl'; main()}
>           x.report {require 'second_impl'; main()}
>         end
> 
> but this seems kludgy and loading the thing at each run could mess up 
> the results, I think.

I don't know.  Does the benchmark package take startup time into
account?  You can always reverse the tests to ensure that they aren't
being tainted by startup time couldn't you?

Another thing you can do is to run your implementation through the
profiler.  I also see ruby-prof on the RAA which you could try.

I also see YABM, but there are no links so I can't comment on that
package.

Dan