From: Gregory Brown <gregory.t.brown@...>
Date: 2009-02-02T10:29:12+09:00
Subject: Minor 1.8 <-> 1.9 performance comparison gotcha

Excited and inspired by the Ruby 1.9.1 release, I wrote a blog post
the other day talking about how much faster Prawn's examples and tests
run on Ruby 1.9 out of the box.  Of course, this isn't scientific
benchmarking but it seemed like a reasonable way to say "Across the
board, things run much faster".

As it turns out, there was a fatal flaw in my (admittedly
unscientific) analysis.  It turned out that my Ruby 1.8.6 was built
without any compile time optimization, but my Ruby 1.9 build had been
set to -O2 .  The difference was HUGE and completely skewed my
results.

This is almost certainly a silly mistake that most people who have
experience comparing manually compiled software would not make, but
I'm sure it'll trip up some folks who don't know what to look for. So
the lesson I learned was: If you're doing performance comparisons,
either to share with the community or for your own needs, be sure to
check your CFLAGS.

 $ ruby -e "require 'rbconfig'; puts Config::CONFIG['CFLAGS']"
 $ ruby19 -e "require 'rbconfig'; puts Config::CONFIG['CFLAGS']"

If the optimization levels don't match, all bets are off :)

You can see the before[0] and after[1] if you wish, otherwise, sorry
to spam those in the know.
Hopefully this quick warning will be helpful to those who might not
have considered it otherwise.

-greg

PS: Jamis Buck helped me track this issue down, so if he's reading: Thanks!

[0] http://blog.majesticseacreature.com/archives/2009.01/prawn_and_ruby19.html
[1] http://blog.majesticseacreature.com/archives/2009.02/lies_and_statistics.html