From: Charles O Nutter Date: 2006-08-02T04:03:20+09:00 Subject: Re: For performance, write it in C - Part 3, Source code now available ------=_Part_52295_32478633.1154458995545 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline And for the record, here are the single-run timings for Java 6 (rather than the same test six times in process): With internal string buffer: headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java Latin Took 617 ms real 0m0.754s user 0m0.664s sys 0m0.044s Without strings at all: headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java Latin Took 368 ms real 0m0.494s user 0m0.420s sys 0m0.024s On 8/1/06, Charles O Nutter wrote: > > The Java code was broken in many different ways, such that any numbers > generated using this code are badly skewed against Java. You can't run > benchmarks against Java, to prove Java's slow, and then write code that's > obviously crippling it. I'll only fix the blatant mistakes here...others > may > do a deeper cleanup of the Java code if they wish. I'll post my code if > requested, but any Java programmer will understand the optimizations as I > list them below. They're pretty obvious. > > And I don't doubt that C will probably be faster, even with those > optimizations...but it won't be faster by much and certainly not by orders > of magnitude. Algorithms where Java does especially well are any that > involve memory allocation, which doesn't come into play here but which is > applicable to almost all real-world code. The point of this thread is that > using the underlying platform code--C for Ruby, Java for JRuby--will often > help many algorithms...and this much is true. But don't venture into > comparing C against Java if you're going to make blanket statements using > flawed tests. > > First, some notes on benchmarking: > > - NEVER include IO when benchmarking a numeric algorithm; IO speeds vary > greatly from system to system and can vary from run to run depending on > what > else is happening > - Do not include initialization code in benchmarks, especially in this > case > where you're manually tweaking a gigantic array > - If you're building up a large chunk of strings, write to an internal > buffer and then write out at the end; don't write for every little tiny > operation. At the very least, use a buffer per-line, rather than a > separate > write for every element on that line. > - Make sure you're actually testing the same thing on all systems; in this > case, the Java code was severely crippled on a number of levels > - I have not changed the algorithm in any way, but an iterative algorithm > would perform a lot better on all platforms. > > So I made some mild optimizations: > > - You do not need to compare boolean values to "true" or "false"; just use > them directly as the test condition. > - Write strings to an internal buffer or do not write them at all; to > support Unicode across platforms, Java normalizes text to the base > platform's preferred encoding, and so incurs extra overhead for this > benchmark than the other versions. If you want to make this a better test, > have the C version use wide char strings internally and normalize to ASCII > on write. > - I moved the initialization of the Compared array to a separate function > and excluded it from the test. I clear out and reinit the string buffer > and > the compared array for each test run. The C code loads a static array into > memory in probably microseconds, so including this initialization for the > Java test totally skews results. > - I had the test benchmark just the call to addRow, since the Java > platform > overhead is a fixed cost outside of this test. If you want to figure that > cost in, you're welcome to...I've left the timings as in the original. > - I ran the algorithm six times per test to allow the JVM to optimize it. > Note how quickly the speed improves once HotSpot gets to it. > > And one caveat: I don't have perl set up properly on this machine, so I > wasn't able to generate the header or run the C code. When I do I'll post > those numbers for comparison. > > Ubuntu Linux 6 (64-bit), current supported kernel (something 2.6.15ish) > Opteron 150, 2.6GHz, 2GB RAM > > All Java versions are AMD64. > > Java 5, client vm, no string creation/buffering: > headius@opteron:~/latin_in_java$ time java Latin > Took 1082 ms > Took 645 ms > Took 388 ms > Took 385 ms > Took 551 ms > Took 385 ms > > real 0m3.667s > user 0m3.436s > sys 0m0.032s > > Java 5, client vm, write strings to internal string buffer: > headius@opteron:~/latin_in_java$ time java Latin > Took 631 ms > Took 599 ms > Took 492 ms > Took 496 ms > Took 492 ms > Took 499 ms > > real 0m3.340s > user 0m3.080s > sys 0m0.116s > > Java 6, client vm, no strings: > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > Latin > Took 400 ms > Took 400 ms > Took 395 ms > Took 408 ms > Took 369 ms > Took 367 ms > > real 0m2.459s > user 0m2.368s > sys 0m0.032s > > Java 6, client vm, write strings to internal buffer: > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > Latin > Took 531 ms > Took 497 ms > Took 478 ms > Took 478 ms > Took 486 ms > Took 494 ms > > real 0m3.172s > user 0m2.940s > sys 0m0.104s > > On 8/1/06, Charles O Nutter wrote: > > > > Ok, so there's a bunch of problems with the Java version. > > > > - In addition to the addRow run and the Java startup time you're also > > benchmarking over 5200 array modifications to set Compared values to > true > > - Your variable naming is entirely contrary to every Java coding > > convention > > published (not a benchmark thing, but it sets off any Java devs warning > > flags) > > - Almost all of the time spent running is spent building and printing > > strings > > > > Benchmarking just the algorithm run itself with no gross string creation > > and > > printing, I'm getting in the neighborhood of 370ms per invocation once > > HotSpot has optimized the code. I'll have more detailed numbers shortly. > > > > -- > > Contribute to RubySpec! @ www.headius.com/rubyspec > > Charles Oliver Nutter @ headius.blogspot.com > > Ruby User @ ruby.mn > > JRuby Developer @ www.jruby.org > > Application Architect @ www.ventera.com > > > > > > > -- > Contribute to RubySpec! @ www.headius.com/rubyspec > Charles Oliver Nutter @ headius.blogspot.com > Ruby User @ ruby.mn > JRuby Developer @ www.jruby.org > Application Architect @ www.ventera.com > > -- Contribute to RubySpec! @ www.headius.com/rubyspec Charles Oliver Nutter @ headius.blogspot.com Ruby User @ ruby.mn JRuby Developer @ www.jruby.org Application Architect @ www.ventera.com ------=_Part_52295_32478633.1154458995545--