From: Charles O Nutter Date: 2006-08-02T04:33:00+09:00 Subject: Re: For performance, write it in C - Part 3, Source code now available ------=_Part_52888_13578395.1154460775618 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline And the C version, which is quite a bit slower on my system than the Java version. Peter, did you confirm both of these are actually running correctly? I didn't do any correctness check...I just fixed what was broken. I'll investigate a bit more as well. Without string writes: headius@opteron:~/latin_in_c$ perl gen.pl 5 > latin.h headius@opteron:~/latin_in_c$ gcc -o latin latin.c headius@opteron:~/latin_in_c$ time ./latin real 0m1.536s user 0m1.524s sys 0m0.004s With string writes: ... headius@opteron:~/latin_in_c$ time ./latin > /dev/null 2>&1 real 0m1.955s user 0m1.800s sys 0m0.016s On 8/1/06, Charles O Nutter wrote: > > And for the record, here are the single-run timings for Java 6 (rather > than > the same test six times in process): > > With internal string buffer: > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > Latin > Took 617 ms > > real 0m0.754s > user 0m0.664s > sys 0m0.044s > > Without strings at all: > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > Latin > Took 368 ms > > real 0m0.494s > user 0m0.420s > sys 0m0.024s > > > On 8/1/06, Charles O Nutter wrote: > > > > The Java code was broken in many different ways, such that any numbers > > generated using this code are badly skewed against Java. You can't run > > benchmarks against Java, to prove Java's slow, and then write code > that's > > obviously crippling it. I'll only fix the blatant mistakes here...others > > may > > do a deeper cleanup of the Java code if they wish. I'll post my code if > > requested, but any Java programmer will understand the optimizations as > I > > list them below. They're pretty obvious. > > > > And I don't doubt that C will probably be faster, even with those > > optimizations...but it won't be faster by much and certainly not by > orders > > of magnitude. Algorithms where Java does especially well are any that > > involve memory allocation, which doesn't come into play here but which > is > > applicable to almost all real-world code. The point of this thread is > that > > using the underlying platform code--C for Ruby, Java for JRuby--will > often > > help many algorithms...and this much is true. But don't venture into > > comparing C against Java if you're going to make blanket statements > using > > flawed tests. > > > > First, some notes on benchmarking: > > > > - NEVER include IO when benchmarking a numeric algorithm; IO speeds vary > > greatly from system to system and can vary from run to run depending on > > what > > else is happening > > - Do not include initialization code in benchmarks, especially in this > > case > > where you're manually tweaking a gigantic array > > - If you're building up a large chunk of strings, write to an internal > > buffer and then write out at the end; don't write for every little tiny > > operation. At the very least, use a buffer per-line, rather than a > > separate > > write for every element on that line. > > - Make sure you're actually testing the same thing on all systems; in > this > > case, the Java code was severely crippled on a number of levels > > - I have not changed the algorithm in any way, but an iterative > algorithm > > would perform a lot better on all platforms. > > > > So I made some mild optimizations: > > > > - You do not need to compare boolean values to "true" or "false"; just > use > > them directly as the test condition. > > - Write strings to an internal buffer or do not write them at all; to > > support Unicode across platforms, Java normalizes text to the base > > platform's preferred encoding, and so incurs extra overhead for this > > benchmark than the other versions. If you want to make this a better > test, > > have the C version use wide char strings internally and normalize to > ASCII > > on write. > > - I moved the initialization of the Compared array to a separate > function > > and excluded it from the test. I clear out and reinit the string buffer > > and > > the compared array for each test run. The C code loads a static array > into > > memory in probably microseconds, so including this initialization for > the > > Java test totally skews results. > > - I had the test benchmark just the call to addRow, since the Java > > platform > > overhead is a fixed cost outside of this test. If you want to figure > that > > cost in, you're welcome to...I've left the timings as in the original. > > - I ran the algorithm six times per test to allow the JVM to optimize > it. > > Note how quickly the speed improves once HotSpot gets to it. > > > > And one caveat: I don't have perl set up properly on this machine, so I > > wasn't able to generate the header or run the C code. When I do I'll > post > > those numbers for comparison. > > > > Ubuntu Linux 6 (64-bit), current supported kernel (something 2.6.15ish) > > Opteron 150, 2.6GHz, 2GB RAM > > > > All Java versions are AMD64. > > > > Java 5, client vm, no string creation/buffering: > > headius@opteron:~/latin_in_java$ time java Latin > > Took 1082 ms > > Took 645 ms > > Took 388 ms > > Took 385 ms > > Took 551 ms > > Took 385 ms > > > > real 0m3.667s > > user 0m3.436s > > sys 0m0.032s > > > > Java 5, client vm, write strings to internal string buffer: > > headius@opteron:~/latin_in_java$ time java Latin > > Took 631 ms > > Took 599 ms > > Took 492 ms > > Took 496 ms > > Took 492 ms > > Took 499 ms > > > > real 0m3.340s > > user 0m3.080s > > sys 0m0.116s > > > > Java 6, client vm, no strings: > > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > > Latin > > Took 400 ms > > Took 400 ms > > Took 395 ms > > Took 408 ms > > Took 369 ms > > Took 367 ms > > > > real 0m2.459s > > user 0m2.368s > > sys 0m0.032s > > > > Java 6, client vm, write strings to internal buffer: > > headius@opteron:~/latin_in_java$ time /usr/lib/jvm/jdk1.6.0/jre/bin/java > > Latin > > Took 531 ms > > Took 497 ms > > Took 478 ms > > Took 478 ms > > Took 486 ms > > Took 494 ms > > > > real 0m3.172s > > user 0m2.940s > > sys 0m0.104s > > > > On 8/1/06, Charles O Nutter wrote: > > > > > > Ok, so there's a bunch of problems with the Java version. > > > > > > - In addition to the addRow run and the Java startup time you're also > > > benchmarking over 5200 array modifications to set Compared values to > > true > > > - Your variable naming is entirely contrary to every Java coding > > > convention > > > published (not a benchmark thing, but it sets off any Java devs > warning > > > flags) > > > - Almost all of the time spent running is spent building and printing > > > strings > > > > > > Benchmarking just the algorithm run itself with no gross string > creation > > > and > > > printing, I'm getting in the neighborhood of 370ms per invocation once > > > HotSpot has optimized the code. I'll have more detailed numbers > shortly. > > > > > > -- > > > Contribute to RubySpec! @ www.headius.com/rubyspec > > > Charles Oliver Nutter @ headius.blogspot.com > > > Ruby User @ ruby.mn > > > JRuby Developer @ www.jruby.org > > > Application Architect @ www.ventera.com > > > > > > > > > > > > -- > > Contribute to RubySpec! @ www.headius.com/rubyspec > > Charles Oliver Nutter @ headius.blogspot.com > > Ruby User @ ruby.mn > > JRuby Developer @ www.jruby.org > > Application Architect @ www.ventera.com > > > > > > > -- > Contribute to RubySpec! @ www.headius.com/rubyspec > Charles Oliver Nutter @ headius.blogspot.com > Ruby User @ ruby.mn > JRuby Developer @ www.jruby.org > Application Architect @ www.ventera.com > > -- Contribute to RubySpec! @ www.headius.com/rubyspec Charles Oliver Nutter @ headius.blogspot.com Ruby User @ ruby.mn JRuby Developer @ www.jruby.org Application Architect @ www.ventera.com ------=_Part_52888_13578395.1154460775618--