From: Charles O Nutter Date: 2006-08-02T03:37:14+09:00 Subject: Re: For performance, write it in C - Part 3, Source code now available ------=_Part_51731_1306357.1154457430710 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline The Java code was broken in many different ways, such that any numbers generated using this code are badly skewed against Java. You can't run benchmarks against Java, to prove Java's slow, and then write code that's obviously crippling it. I'll only fix the blatant mistakes here...others may do a deeper cleanup of the Java code if they wish. I'll post my code if requested, but any Java programmer will understand the optimizations as I list them below. They're pretty obvious. And I don't doubt that C will probably be faster, even with those optimizations...but it won't be faster by much and certainly not by orders of magnitude. Algorithms where Java does especially well are any that involve memory allocation, which doesn't come into play here but which is applicable to almost all real-world code. The point of this thread is that using the underlying platform code--C for Ruby, Java for JRuby--will often help many algorithms...and this much is true. But don't venture into comparing C against Java if you're going to make blanket statements using flawed tests. First, some notes on benchmarking: - NEVER include IO when benchmarking a numeric algorithm; IO speeds vary greatly from system to system and can vary from run to run depending on what else is happening - Do not include initialization code in benchmarks, especially in this case where you're manually tweaking a gigantic array - If you're building up a large chunk of strings, write to an internal buffer and then write out at the end; don't write for every little tiny operation. At the very least, use a buffer per-line, rather than a separate write for every element on that line. - Make sure you're actually testing the same thing on all systems; in this case, the Java code was severely crippled on a number of levels - I have not changed the algorithm in any way, but an iterative algorithm would perform a lot better on all platforms. So I made some mild optimizations: - You do not need to compare boolean values to "true" or "false"; just use them directly as the test condition. - Write strings to an internal buffer or do not write them at all; to support Unicode across platforms, Java normalizes text to the base platform's preferred encoding, and so incurs extra overhead for this benchmark than the other versions. If you want to make this a better test, have the C version use wide char strings internally and normalize to ASCII on write. - I moved the initialization of the Compared array to a separate function and excluded it from the test. I clear out and reinit the string buffer and the compared array for each test run. The C code loads a static array into memory in probably microseconds, so including this initialization for the Java test totally skews results. - I had the test benchmark just the call to addRow, since the Java platform overhead is a fixed cost outside of this test. If you want to figure that cost in, you're welcome to...I've left the timings as in the original. - I ran the algorithm six times per test to allow the JVM to optimize it. Note how quickly the speed improves once HotSpot gets to it. And one caveat: I don't have perl set up properly on this machine, so I wasn't able to generate the header or run the C code. When I do I'll post those numbers for comparison. Ubuntu Linux 6 (64-bit), current supported kernel (something 2.6.15ish) Opteron 150, 2.6GHz, 2GB RAM All Java versions are AMD64. Java 5, client vm, no string creation/buffering: headius@opteron:~/latin_in_java$ time java Latin Took 1082 ms Took 645 ms Took 388 ms Took 385 ms Took 551 ms Took 385 ms real 0m3.667s user 0m3.436s sys 0m0.032s Java 5, client vm, write strings to internal string buffer: headius@opteron:~/latin_in_java$ time java Latin Took 631 ms Took 599 ms Took 492 ms Took 496 ms Took 492 ms Took 499 ms real 0m3.340s user 0m3.080s sys 0m0.116s Java 6, client vm, no strings: headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java Latin Took 400 ms Took 400 ms Took 395 ms Took 408 ms Took 369 ms Took 367 ms real 0m2.459s user 0m2.368s sys 0m0.032s Java 6, client vm, write strings to internal buffer: headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java Latin Took 531 ms Took 497 ms Took 478 ms Took 478 ms Took 486 ms Took 494 ms real 0m3.172s user 0m2.940s sys 0m0.104s On 8/1/06, Charles O Nutter wrote: > > Ok, so there's a bunch of problems with the Java version. > > - In addition to the addRow run and the Java startup time you're also > benchmarking over 5200 array modifications to set Compared values to true > - Your variable naming is entirely contrary to every Java coding > convention > published (not a benchmark thing, but it sets off any Java devs warning > flags) > - Almost all of the time spent running is spent building and printing > strings > > Benchmarking just the algorithm run itself with no gross string creation > and > printing, I'm getting in the neighborhood of 370ms per invocation once > HotSpot has optimized the code. I'll have more detailed numbers shortly. > > -- > Contribute to RubySpec! @ www.headius.com/rubyspec > Charles Oliver Nutter @ headius.blogspot.com > Ruby User @ ruby.mn > JRuby Developer @ www.jruby.org > Application Architect @ www.ventera.com > > -- Contribute to RubySpec! @ www.headius.com/rubyspec Charles Oliver Nutter @ headius.blogspot.com Ruby User @ ruby.mn JRuby Developer @ www.jruby.org Application Architect @ www.ventera.com ------=_Part_51731_1306357.1154457430710--