From: Ezra Zygmuntowicz <ezmobius@...>
Date: 2008-12-02T04:17:26+09:00
Subject: Re: Some benchmarks with MinGW, JRuby, MSVC...


Benchmarking ruby on windows is fairly useless. MRI 1.8.* is known to  
be very slow on windows and harldy anyone deploys production apps to  
windows because of this. If you want your benches to be taken  
seriously you need to run them on linux, bsd or osx where ruby  
performs much better. Also make sure to compile ruby with --disable- 
pthreads for best perf.


Cheers-
Ezra Zygmuntowicz
ez@engineyard.com




On Nov 30, 2008, at 9:14 AM, Joao Pedrosa wrote:

> So, I have managed to build several different versions of Ruby to
> compare all
> in some benchmarks. I am going to let the numbers talk for themselves.
> It was
> all tested on an old desktop computer with Windows XP, so I am not
> testing
> anything too impressive.
>
> Of importance is the seemingly poor performance of Ruby 1.8.7 for
> whatever
> reason. It's as if Ruby 1.8.7 were like Windows ME, a version between
> major
> versions, as an analogy. :-)
>
> Also, I was not able to run the Mongrel gem on Ruby 1.9.1.
>
> I am going to order the data by fastest to make it easier to compare
> it on text.
> I have collected the data on a Google Spreadsheet:
> http://spreadsheets.google.com/ccc?key=pmtaDyNoButGKeSVdIGi7Fg
>
> I am going to keep it short here and for more detail on versions and
> whatnot
> please refer to the spreadsheet.
>
> == WEBrick (req/s)
>
> 22.39 - jruby
> 16.95 - jruby --server
> 12.82 - ruby oneclick 1.8.6
> 12.49 - ruby msvc 1.8.6
> 12.45 - ruby rubyinstaller 1.8.6
> 12.43 - ruby mingw 1.8.6
> 11.94 - ruby msvc 1.9.1
> 11.48 - ruby mingw 1.9.1
> 3.42 - ruby msvc 1.8.7
> 3.40 - ruby mingw 1.8.7
>
>
> == Mongrel (req/s)
>
> 357.14 - ruby mingw 1.8.6
> 347.45 - ruby rubyinstaller 1.8.6
> 326.70 - ruby msvc 1.8.6
> 313.57 - ruby oneclick 1.8.6
> 274.68 - ruby mingw 1.8.7
> 232.39 - ruby msvc 1.8.7
> 71.24 - jruby
> 62.22 - jruby --server
> (crash when starting up) - ruby mingw 1.9.1
> (error when building gem) - ruby msvc 1.9.1
>
>
> == bench_pythag.rb (s)
>
> 19.09 - jruby --server
> 24.35 - jruby
> 31.72 - ruby mingw 1.9.1
> 31.91 - ruby mingw 1.8.6
> 32.73 - ruby mingw 1.8.7
> 33.94 - ruby msvc 1.9.1
> 35.67 - ruby rubyinstaller 1.8.6
> 35.83 - ruby msvc 1.8.7
> 39.79 - ruby msvc 1.8.6
> 48.30 - ruby oneclick 1.8.6
>
>
> == bench_fractal.rb (s)
>
> 3.22 - jruby --server
> 3.89 - jruby
> 5.67 - ruby msvc 1.9.1
> 5.75 - ruby mingw 1.9.1
> 9.80 - ruby mingw 1.8.6
> 9.90 - ruby mingw 1.8.7
> 10.11 - ruby rubyinstaller 1.8.6
> 10.30 - ruby msvc 1.8.6
> 10.64 - ruby msvc 1.8.7
> 15.17 - ruby oneclick 1.8.6
>
>
> == bench_float_math.rb (s)
>
> 103.44 - jruby --server
> 125.89 - jruby
> 158.64 - ruby msvc 1.9.1
> 175.41 - ruby mingw 1.9.1
> 176.61 - ruby mingw 1.8.6
> 179.94 - ruby mingw 1.8.7
> 184.73 - ruby rubyinstaller 1.8.6
> 195.37 - ruby msvc 1.8.7
> 197.39 - ruby msvc 1.8.6
> 279.62 - ruby oneclick 1.8.6
>
>
> Since I collected these data, the Ruby 1.9.1 branch has had a number
> of updates
> for one thing. :-)
>
> Benchmarks can be kind of tough to do right and there are all
> possibilities when it
> comes to doing them.
>
> If anyone has an algorithm that he would like to test on all those
> Ruby implementations
> I would be glad to test it. With the exception of testing Rails, which
> I have never used
> and don't know enough about it to willingly make myself available when
> testing it. :-)
>
> Cheers,
> Joao
>