From: Eric Mahurin <eric.mahurin@...>
Date: 2006-08-01T08:40:52+09:00
Subject: Re: converting some autogenerated ruby code to C

On 7/31/06, Dominik Bathon <dbatml@gmx.de> wrote:
> > What you have sounds quite robust and able to handle a wide variety of
> > ruby functionality.  Unfortunately, it still doesn't help much on the
> > performance front :(   1-1.5X just doesn't seem like enough to justify
> > the effort.  The main problem looks to be rb_funcall*.
>
> Yes, rb_funcall seems to be slow. That's exactly what the public builtin
> methods optimization tries to address, it replaces the rb_funcall with
> (almost) only two C calls. Have you tried the svn version?

Cool.  I have not tried any of your stuff yet.  I'm not at my ruby
development machine right now.

> > Out of curiousity, could this be used with meta-classes?  I do
> > something like this to get self-defining methods:
> >
> >         def test(a)
> >             (class << self;self;end).class_eval( "
> >                 def test(a)
> >                     #{tester("a")}
> >                 end
> >             " )
> >             test
> >         end
> >         def tester(a)
> >             %Q{puts("hello " + #{a})}
> >         end
> >
> > If I were to go down the C optimization route, I would want the
> > ability to convert the generated ruby (from #tester) to C and
> > compile/load it for this object.  This is the only place I really want
> > a lot of optimization.  I'm already doing quite a bit in my ruby code
> > generation and have more to go.
> >
> > If possible and you don't do it already, you might want to think about
> > ruby2cext replacement methods (different names) for all of the eval
> > methods that work on strings/files: eval, instance_eval, class_eval,
> > require, etc.  When a compiler isn't available on the system or
> > something goes wrong, you could revert to the original eval methods.
>
> I think it is hard to do this transparently and efficient at the same
> time. You would either have to generate and compile and load a C extension
> for _each_ call of *eval or somehow collect all the code to eval and do it
> all at once (but then it probably can't be done transparently).
>
> Maybe you can generate code like:
>
> module TestDefiner
>    def self.define_tests(instances)
>      (class << instances[0];self;end).class_eval {
>        def test(a)
>          puts("hello" + a)
>        end
>        def test2(...)
>          ...
>        end
>        ...
>      }
>      (class << instances[1];self;end).class_eval {
>        def test3(...)
>          ...
>        end
>        def test4(...)
>          ...
>        end
>        ...
>      }
>      ...
>    end
> end
>
> Then compile that at once to an extension, require it and call (from Ruby
> code): TestDefiner.define_tests([instance1, instance2, ...])
>
> This would require some changes in your code, but I think that would be
> the way to go once you have decided that compiling to C is worth it.

Unfortunately this won't work.  In the above example, I just threw
something arbitrary together.  I have something where the string that
#tester generates isn't determined until runtime and is dependent on
the object hierarchy.  In my code the methods are named scan/scanner
instead of test/tester.  I use this mechanism for making
macros/flattening methods.  My parsers (the #scan method) end up being
one giant expression (if there is no recursion).  I'm flattening to
get rid of as much of the method calling overhead as I can.

It still should be possible to load a C extension into a meta-class,
right?  Wouldn't the Init_* function just define the method in the
meta-class instead of a plain class?  Maybe you'd use a temp global
variable to pass this meta-class.  I realize caching these so that you
don't have to recompile could be an issue.  You could use md5 or
whatever (even string length) to get a mostly unique identifier for
the c/so (like rubyinline).