From: Caleb Clausen Date: 2006-08-11T04:34:45+09:00 Subject: Re: Been there, done that - language shootout On 8/10/06, Isaac Gouy wrote: > Caleb Clausen wrote: > > (Provided we undestand that this benchmark is testing the performance > > of an arbitrary algorithm, not the performance that a real program > > using random numbers in a given language would experience.) > > Maybe you could explain a little more - if someone was using one of > those programs to generate Fasta files for program testing and > algorithm development in their lab, would that make it a real program > rather than an arbitrary algorithm? What do you say the difference is > between a real program and an arbitrary algorithm? My understanding of the goal of the fasta program(s) (aside from being a benchmark) is to generate a random dna sequence, possibly with some biases about which symbols are more likely. If I were really using such a program in my work, and speed in that program was important to me, (and I was writing it in Ruby,) I would not use the version you give on your website. I would make use of Ruby's already existing random number generator. (Among other things, ruby uses (AFAIK) the well-regarded Mersenne twister algorithm, whereas you have specified a linear congruential generator, which is considered inadequate for hard-core random number applications.) For benchmarking purposes, what you have already is just fine, provided you understand the limitations. I understand the need to have all the languages implement the same algorithm so that you can have an apple-to-apples comparison. I can also sympathise with your desire to keep the testing simpler by requiring a specific output. It's just that this isn't the way a serious programmer would actually use random numbers in Ruby (and probably most other languages). What you are really testing is not the speed at which ruby generates random numbers, as they would be used in the real world, but the speed at which it can execute the linear congruential algorithm, which ultimately depends on the speed at which it can execute integer math operations.