From: Rick DeNatale Date: 2009-08-20T21:20:29+09:00 Subject: Re: Confirm my Performance Test Against Java? On Wed, Aug 19, 2009 at 10:41 AM, wrote: > On Wed, Aug 19, 2009 at 9:31 AM, Ben Christensen > wrote: >> I'm evaluating Ruby for use in a variety of systems that are planned by >> default to be Java. >> >> I've started down a path of doing various performance tests to see what >> kind of impact will occur by using Ruby and in my first test the numbers >> are very poor - so poor that I have to question if I'm doing something >> wrong. > > Is this test case in any way representative of the tasks you will > actually be performing? > > Test file 1: > >> java FileReadParse > Starting to read file... > The number of tokens is: 1954 > It took 16 ms > >> ruby -v file_read_parse.rb > ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-linux] > Starting to read file ... > The number of tokens is: 1954 > It took 4.951 ms > > Test file 2: > >> java FileReadParse > Starting to read file... > The number of tokens is: 479623 > It took 337 ms > >> ruby file_read_parse.rb > Starting to read file ... > The number of tokens is: 479623 > It took 2526.455 ms > >> ruby file_read_parse-2.rb > Starting to read file ... > It took 588.065 ms > The number of tokens is: 479623 One of the things this 'benchmark' skips over is the time it takes to initialize the two environments. There's no measurement of the time between hitting enter on the command line and the point where the start = Time.now line (or it's equivalent in the Java program) gets executed. It might be interesting to execute those benchmarks using something like the linux time command to measure the differences in "womb to tomb" execution times. I suspect, but I may be totally wrong, that Java takes a while to 'warm up' a more complex runtime environment, and that Ruby gets going faster. The JVM has evolved in an environment where it has tended to be used for long-running processes. Depending on the task this may be important. This can also affect deployment architecture. JRuby tends to encourage multi-threading in a single process to amortize the start-up time. Most of us who are using MRI for, say rails, are pretty happy with having multiple server processes which can be brought up as needed (and terminated when not) under something like Passenger (a.k.a. mod ruby) particularly using the "Enterprise" version of Ruby which allows for sharing vm code between the processes. -- Rick DeNatale Blog: http://talklikeaduck.denhaven2.com/ Twitter: http://twitter.com/RickDeNatale WWR: http://www.workingwithrails.com/person/9021-rick-denatale LinkedIn: http://www.linkedin.com/in/rickdenatale