From: Brian Candler <B.Candler@...>
Date: 2003-04-04T04:03:33+09:00
Subject: Re: Ruby 1.6.8 vs Ruby 1.8.0 preview 2 - benchmarks

On Sat, Mar 29, 2003 at 02:04:16PM +0900, Daniel Berger wrote:
> One of the concerns I'm having about Ruby 1.8 is that it appears to be
> getting *slower*.  Not so much that it will be a serious issue for me,
> but Ruby's speed has been something some of the Perl and Python folks
> have been using against Ruby.  I visited the Great Computer Language
> Shootout (http://www.bagley.org/~doug/shootout/), put together 9 of
> the functions he used and ran them using both 1.6.8 and 1.8 p2.
> 
> In 7 of the 9 tests, 1.8 was *slower*.  Should I be worried?

Just out of interest, I ran your tests on my system (FreeBSD-4.7, P266MMX,
128MB RAM) and the results are below. I installed the 16->18 shim to get the
benchmark module.

Looking at the "total time" column they are both pretty similar, with 1.8
faster in five of the tests, very much so in the case of 'Lists', and only
significantly slower in two (Hash Access I and Word Frequency)

For some reason the "real" time column is slower. e.g.
    Array access:  total user+cpu time:  17.22 ---> 16.95
                   real time:            17.63 ---> 17.97

There must be something funny going on - the program does the tiniest amount
of I/O so it's hard to see what it's blocking on. I think it may be because
of the following warning which is generated twice for each test:

/usr/local/lib/ruby/site_ruby/1.8/benchmark.rb:435: warning: obsolete method Time::times; use Process::times

It probably makes more sense to compare the CPU used rather than the actual
absolute execution time anyway.

Regards,

Brian.


[brian@vaio ruby]$ ruby -v benchmark.rb
ruby 1.6.8 (2003-01-27) [i386-freebsd4.7]
      user     system      total        real
Ackermann function:   2.500000   0.000000   2.500000 (  2.656021)
Array access:  17.203125   0.015625  17.218750 ( 17.631144)
Fibonacci numbers:  16.625000   0.007812  16.632812 ( 16.901553)
Hash access I:  14.890625   0.007812  14.898438 ( 15.144399)
Hash access II:  19.945312   0.031250  19.976562 ( 20.313897)
Lists:  55.296875   0.039062  55.335938 ( 56.143935)
Nested loop:  18.265625   0.000000  18.265625 ( 18.525846)
Sieve of Eratosthenes:  27.195312   0.054688  27.250000 ( 27.721882)
Word Frequency:   5.726562   0.007812   5.734375 (  5.821429)

[brian@vaio ruby]$ ruby18 -v benchmark.rb
ruby 1.8.0 (2003-03-03) [i386-freebsd4.7]
      user     system      total        real
Ackermann function:   2.578125   0.000000   2.578125 (  2.972841)
Array access:  16.921875   0.023438  16.945312 ( 17.969264)
Fibonacci numbers:  15.703125   0.015625  15.718750 ( 16.443847)
Hash access I:  17.484375   0.015625  17.500000 ( 18.225830)
Hash access II:  19.109375   0.054688  19.164062 ( 19.871894)
Lists:   4.304688   0.015625   4.320312 (  4.715083)
Nested loop:  18.023438   0.007812  18.031250 ( 18.852277)
Sieve of Eratosthenes:  23.718750   0.093750  23.812500 ( 24.495048)
Word Frequency:   6.773438   0.023438   6.796875 (  7.650401)