About efficiency

Hello,

Short Version:

I am looking for tips about writing more efficient Ruby programs. I have 
found http://www.bagley.org/~doug/shootout/lang/ruby/ and not much more. I 
guess I could have a deeper look at the MRI C interpreter, but that seems a 
bit extreme.

Thanks,

JHR

-------------
Long Version:

I have been enjoying Ruby for 2 weeks now. I am very impressed. It is my 
understanding that currently the interpreter directly executes the parse 
tree versus going thru some bytecode generation phase (the later is what I 
did for some interpreter I wrote years ago, while at the same time a friend 
of mine was going the ruby way, hence we had opportunities to compare).

I don't have a definitive answer about which approach is faster (hints 
welcome) and I guess Matz has some good reasons to plan for some bytecode 
generation (besides doing what Java does...). What matters more to me is 
that it is a clear indication that we all care about efficiency to some extend.

Yet, I have a hard time figuring out what is efficient versus what is not 
in Ruby. I am not talking about what algorithm I pick, this is language 
independent mostly.

Here are a few questions:

1) Would using Symbol more be more efficient ?
While writing some debugging tool, I was surprised to discover that a 
function like Kernel.set_trace_func() would set a proc that is given a 
"String" to describe what is happening. I would have guessed that a Symbol 
would obviously be more efficient here (comparing two symbols is like 
comparing two pointers, whereas comparing two strings may involve comparing 
byte per byte). Maybe it's not a big deal. But these are the deals that end 
up making things go faster I think.

Migrating toward more use of Symbol is not very difficult, specially if 
using Symbol (instead of String, when it makes sense) gets some help from 
the language (i.e. like "aaa" == :aaa resulting in true instead of false 
today).

2) "xxx" versus 'xxx', which is more efficient ?
Equivalent according to Matz's recent comment. Well... I guess that when 
xxx does not include any special construction, one of the two *has* to be 
faster than the other one, either at compile time (parse tree constuction 
time I mean) or at runtime, right ? Which one ? I am used to using "xxx" 
but if 'xxx' is more efficient, I can use it instead (when both are 
semantically equivalent).

3) closure
I suspect that holding a closure means at least that the stack frame where 
the closure was created is referenced (and not garbage collected until the 
closure disappears) (I hope that the caller stack frames aren't referenced, 
are they ?). This may have big memory impacts, specially if the stack frame 
also holds references to large object. For sure it does not hurt to do some 
xxx=nil when xxx is not usefull anymore.

Is there a way to know about that, because it is nowhere in any of the 
book/article I have read so far, as if nobody cared (which is the subject 
this message wishes to address).

Detecting that a block does not need to reference its binding environment 
is probably non trivial (if possible at all), isn't it ? Yet, there are 
solutions, as in C++ where a smart use of "const" helps the compiler to 
understand that it can safely make a copy of what the closure needs, versus 
keeping every thing referenced.

3) type induction
One day the "dynamic typing" versus "static typing" language war will be 
over. That day, you will have the ability to switch smoothly from dynamic 
to static, during the optimization phase of your project. No need for C++ 
style templates then. All you will have to do is give hints to the 
compiler/interpreter about the domain/type/range of variables so that the 
compiler/interpreter can take advantage of that knowledge to optimize the 
code. I have seen that being applied years ago in Turbo Prolog to some 
extend, with incredible results. Any plan for Ruby about that ?

4) String
For a String intensive language like Ruby, there is an optimization that 
could be worth implementing. It is an optimization where a String is 
actually made of a reference to some potentially shared string 
representation, + an_offset, + a_size. That means that many operations then 
do not need to copy the string anymore (you update 'an_offset' and/or 
'a_size' instead, still referencing the same 'string_representation'). Has 
such an optimization been evaluated (it has almost no impacts on the 
external interface) ? I think this is a type of "Copy On Write" optimization.

5) Cache.
Reading the books about the implementation of Smalltalk gives some hints on 
how much efficient such caches can be. Candidates are: caching 
(a_class,a_method_name)=>a_method. This cache alone doubled the speed of 
the smalltalk interpreter !

Another one is for (a_class,an_instance_variable_name)=>an_index. Assuming 
an object has a value (i.e. has some set of instance variables) and if 
accessing that set directly is an option (with an index, i.e. versus 
sequentially or some other less efficient method) then again this can speed 
things up a lot (in the most common cases, where objects have instance 
variables layed out in the same order).

A less trivial cache is for inlining small methods. This is specially 
efficient for accessors. I think it is described too in the books about the 
implementation of Smalltalk 80.

after 7 years of existence, I guess that a profiling of Ruby does not show 
any hot-spot anymore (I remember malloc() beeing a big one in my 
interpreter, before I implemented pools ; strangely enough getimeofday() 
was another one, I had to implement an heuristic to estimate "how often" I 
had to call it to get a fast/reliable measure of quantums between thread 
switches)...

-----
As CPU gets faster, it becomes more and more compelling to use higher level 
languages. Yet, the faster the language, the more chances it has to be 
adopted *now* (versus when CPU's speed permit). I have some expectations 
that Ruby could be the next step, after machine code, C, C++ and Java. The 
faster it runs (without compromising too much about flexibility and 
dynamics), the better (but I guess everybody would agree on that). 
Impressive job !

Any info/links to Ruby's efficiency/tips ?

Thanks,

Jean-Hugues