From: Caleb Clausen <vikkous@...>
Date: 2006-08-11T01:49:25+09:00
Subject: Re: Been there, done that - language shootout

On 8/9/06, Isaac Gouy <igouy@yahoo.com> wrote:
> If you're interested enough to speculate and question, please be
> interested enough to click the URL that was provided and find out
> something about it before you speculate and question ;-)
>
> Here it is again
> http://shootout.alioth.debian.org/gp4/benchmark.php?test=fasta&lang=all#about

All right. I'm looking at it. Based on what I'm seeing on your
website, I'll make the following statements. Correct me if I'm wrong
about any of these:

1) Your fasta benchmark is a random number (err.. DNA sequence)
generator. I feel this is a perfectly legitimate benchmark, by itself.
(Provided we undestand that this benchmark is testing the performance
of an arbitrary algorithm, not the performance that a real program
using random numbers in a given language would experience.)

2) 2 subsequent benchmarks, k-nucleotide and reverse-complement, use
the data generated by fasta. It appears that both are reading that
data from a canned file, and not calling the fasta random number
generator.

3) Therefore, Eric's statement:

> I have to agree with Austin here.  There are benchmarks inside that
> specify "this benchmark must use the implementation of the random
> number generator from benchmark X".

is incorrect. Ruby's (admittedly poor) performance on the fasta
benchmark should not be further artifically be dragging down its
(admittedly poor) performance on the subsequent 2 benchmarks.

4) Going back and looking again, I see in your reply to Eric that you
seem to be making this point. This was not appearent to me on first
reading. You could have been a little clearer.... on the other hand, I
could have done a little more digging before I posted. Still and all,
it looks like Ara came to the same misunderstanding.