From: Austin Ziegler Date: 2005-06-11T05:59:22+09:00 Subject: Re: python/ruby benchmark. On 6/10/05, Isaac Gouy wrote: > Trey Campbell wrote: >> On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote: >>> I personally think that most of the shootout items are bogus in >>> just about every way. Don't get me wrong -- I'm not at all >>> suggesting that Ruby couldn't be faster. I'm just saying that >>> benchmarks, like statistics, are lies. >> The most succinct way that I've heard it put is that there are >> "lies, damn lies, and benchmarks". > There isn't with benchmarks per-se; there can be a problem with > how one chooses to interpret benchmarks. No, it's the benchmarks themselves that are the problem. Remember that NVidia got caught detecting a benchmark in their drivers and optimizing output for the benchmark. Similarly, this thread was started because someone came across your language shootout benchmark page and assumed that Ruby is dramatically slower in Real Life Examples because it's slower in the benchmarks. Which, I can assure you, it most assuredly NOT. You get it correct when you indicate that "the best benchmark is your program." That's the only one that matters. It also lets you measure benchmarks that can't be categorised in CPU seconds or memory use or FLOPS or other nonsense like that. It's hype, but the idea that Rails makes one 10x more productive is part of the value of Ruby. Unless your program is of a special case -- and a lot of these special cases are handled already -- then Ruby programs will be developed faster and more easily maintained than their non-Ruby counterparts. Enough so that execution speed is something of a wash. At any rate, I consider the Alioth shootout to be harmful to all languages involved. There is no useful value provided by it, especially as it does not permit language-appropriate modifications to the algorithms in use. -austin -- Austin Ziegler * halostatue@gmail.com * Alternate: austin@halostatue.ca