From: "M. Edward (Ed) Borasky" <znmeb@...>
Date: 2007-09-24T22:27:17+09:00
Subject: Re: Latest benchmarks

SASADA Koichi wrote:
> Hi,
> 
> M. Edward (Ed) Borasky wrote:
>> I can do fancier stuff in R, but I probably won't -- 5x is easy to remember.
> 
> Some warning:
> 
> 1. Results of vm[12]_* benchmarks should be subtracted
> loop_whileloop[12] results.
> 
> Does anyone hack benchmark program?

Well ... I'm putting together a benchmark suite, including the ones in
Ruby 1.9.0 and my own MatrixBenchmark. And I'm planning to duplicate
(and enhance) the logic in the spreadsheet in Ruby.
> 
> 
> 2. These benchmarks are only "micro" benchmarks.  The purpose of
> these benchmarks are only to measure VM basic performance.
> Especially benchmarks focuses features which YARV achives
> performance improvement on.
> 
> So practical application (such as using String manipulation) will
> not be fast.

Isn't the String manipulation written in C already? It won't be slower,
and overall a program that does a lot of String manipulation will run
faster because its spending more of its time doing String manipulation
and less of its time interpreting the Ruby code *around* the String
manipulation.

In general, yes, micro-benchmarks will have a higher speedup than
real-world programs for a variety of reasons. At the same time, though,
a faster VM helps *everybody*, while faster String operations only help
people doing String processing, faster Bignum operations will help my
Matrix benchmark but not a script that searches for patterns in a huge
text file using Oniguruma, and neither one of them will help a
multimedia codec that's doing a discrete cosine transform.