From: Dick Davies <rasputnik@...>
Date: 2005-04-06T01:46:33+09:00
Subject: Re: Benchmarking whole programs

* Berger, Daniel <Daniel.Berger@qwest.com> [0442 17:42]:
> > -----Original Message-----
> > From: gabriele renzi [mailto:surrender_it@remove-yahoo.it] 
> > Sent: Tuesday, April 05, 2005 10:35 AM
> > To: ruby-talk ML
> > Subject: Benchmarking whole programs
> > 
> > 
> > Hi gurus and nubys,
> > 
> > I was playing with the  ruby programs in the alioth language 
> > shootout and I realized I have no idea on how to benchmark 
> > them all. What I mean is: If I want to benchmark a single 
> > function I can use the 
> > usual benchmark code also shown in ri, but what if I want to 
> > check two 
> > slightly different implementations of a a single program?
> > 
> > I can't just stick them all in a #meth1 and #meth2 thing, since there 
> > may be constants defined (actually they almost surely will, at least 
> > classes and modules), I suppose I could do something like:
> >         Benchmark.bm do |x|
> >           x.report {require 'first_impl'; main()}
> >           x.report {require 'second_impl'; main()}
> >         end
> > 
> > but this seems kludgy and loading the thing at each run could mess up 
> > the results, I think.
> 
> I don't know.  Does the benchmark package take startup time into
> account?  You can always reverse the tests to ensure that they aren't
> being tainted by startup time couldn't you?
> 
> Another thing you can do is to run your implementation through the
> profiler.  I also see ruby-prof on the RAA which you could try.

For those who don't know - 
Don't use that for a walltime, it slows things right down.

For a full app, what's wrong with

time first_impl.rb
time second_impl.rb 

-- 
'Tempers are wearing thin. Let's hope some robot doesn't kill everybody.'
		-- Bender
Rasputin :: Jack of All Trades - Master of Nuns