Re: Confirm my Performance Test Against Java?

Ben,

Thanks for provoking a  productive discussion.
I think we all agree that Ruby is slow, very slow, and my impression  
is that you underestimate the slowdown.

But ...

The "is Ruby fast enough?" discussion suffers from the same flaws as  
many that precede it ( Fortran vs Assembler, C vs Fortran, C++ vs C, C+ 
+ vs Fortran, Java vs C++ ...)

The discussion rests on some faulty assumptions:

That language runtime performance will dictate system performance. In  
fact, rarely is that true and in performance engineering the truth is  
much more farcical than anyone might think...

The first time I was paid to write Fortran code I'd been warned that  
Fortran would be unacceptably slow compared to Assembler. It didn't  
matter. I simply was not capable of writing sophisticated time series  
analysis code in Assembler. In fact a large part of that project was  
built and deployed with GW-Basic on a 6MHz 8086 CPU with 512K of RAM.  
As a newbie programmer I didn't realize that an interpreted language  
could not perform, and the app successfully predicted windshifts in  
real time in about 5% of the time that been budgeted. Perhaps with  
more work experience I would have known better ;-)

Since then I've done a bunch of performance critical coding and, over  
the past few years, a bunch of tuning work.

Ruby's 3x performance penalty is enormous.

But it's dwarfed by the performance degradation caused by typical  
coding and typical physical architectures.

Two real, typical datapoints, from a list of hundreds ...

In Dec 2008 I tuned a production Rails app that had 100,000 users,  
improving the client side build time, for the test page,
from 2.2 sec to 181 ms, (a factor of 12). In an appendix to that  
project I identified more than a dozen unimplemented tunings that  
could further lower that build time to about 7 msec.

In 2003 I worked on a similar Java project, and spent much longer  
tuning a similar dynamic page (running on much slower hardware). The  
team implemented more than 500 performance fixes over six months,  
improving page build times from approx 2.5 sec to 14 ms (a factor of  
180x).

Neither app was built by weak programmers - in fact they were two  
extremely smart development teams.

So when you describe a web service that responds (server side) in 140  
ms, and ask why you should consider a toolset that might triple that  
response time, I ask

"has someone else deployed a similar web service that responds in 2  
seconds?"
"has someone else deployed a similar web service that responds in 2 ms?"

"what would it take for it to respond in 20 ms?
"what would it take for it to respond in 5 ms?"
"what would it take for it to respond in 1 ms?"

  I hate slow code and slow websites and I resent the time I waste  
waiting for both.
  But our industry norm is for system response times to be 100x or  
more slower than they need to be.

You might think "BS", or "OK, but he's talking about the doofus  
programmers, he's not talking about us."

I'm talking about you, me, all of us.

I don't know anything about the web service that you describe but I  
will happily wager $50 that we can take any Java web service that is  
currently running in production, and replace it with a Ruby equivalent  
that is twice as fast.

Note that I'm not saying "I can". I'm saying that you, me or any smart  
programmer here can do this.

Here's the thing - I've worked on at least a dozen platform rewrite  
projects, going back more than 20 years (
"we need a C version of this hand optimized assembler file IO layer"
"Hey build a Java version of this C++/X app",
"we need a web version of this desktop app",
"we need a script version of this compiled app").

Typically there's an accompanying message that management understand  
that it might be twice as slow.

On every single occasion,  the surprising outcome is that the new  
version, built with a higher level "slower" toolset, outperformed the  
"stable, optimized, tuned" version, typically by a factor of 3 or  
more. So I'd be a fool to continue being surprised by this. I'm not  
saying that I'm a better programmer than anyone. I am saying that the  
amount of wasted resources in most deployed systems is much, much  
higher than people realize, for a whole set of reasons.


Thanks for initiating such an interesting conversation, and for  
persisting with it.

Peter Booth.