From: Austin Ziegler <halostatue@...>
Date: 2006-09-15T04:00:38+09:00
Subject: Re: Metaruby, BFTS, Cardinal and Rubicon - State of play?

On 9/14/06, Ryan Davis <ryand-ruby@zenspider.com> wrote:
> On Sep 13, 2006, at 7:59 AM, Austin Ziegler wrote:
> > I'd be happier if Mr Tew didn't try to lend legitimacy to the alioth
> > shootout. Microbenchmarks don't show anything useful, even if they're
> > run correctly -- which the shootout has never been run correctly. It
> > isn't even administered correctly. (I was similarly annoyed that Joel
> > Spolsky used it in his latest slam on Ruby. Stupid, Joel, stupid.)
> Thank you, once again, for derailing a thread with your personal
> vendetta.

Look. We *know* that the shootout is crap. We've known this for three
years now. But we *still* have people come in and use it for a variety
of reasons, most of which are completely bogus.

If we, as Ruby users, want to have a set of benchmarks, we should
develop our own and hold them to a rigorous standard. That is, the
exact *opposite* of what Mr. Guouy's shootout does.

I'd gladly support a Ruby benchmark suite that people could use in
talking about things. But in a variety of different threads we've seen
that not only don't the shootout people know anything about
benchmarks, most other people don't either (see the "For
performance..." thread).

Zed Shaw has had to do a lot of teaching about statistics. I'm sure
that Ed Borasky could teach us a lot about benchmarking with *good*
benchmarks to be tested under controlled or controllable situations so
we could improve the performance of Ruby in various situations.

-austin
-- 
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * http://www.halostatue.ca/feed/
               * austin@zieglers.ca