From: ara.t.howard@...
Date: 2006-02-23T07:52:42+09:00
Subject: Re: Dynamic code generation and linking

On Thu, 23 Feb 2006, Benjohn Barnes wrote:

> I'm very interested in being able to dynamically generate, compile, and link
> in c code (and potentially other languages, including assembly - but c is my
> primary interest), from within Ruby. When I say "link in", I mean have the
> compiled code immediately become available for use by the Ruby script that
> asked for the compilation.
>
> So, I'm imagining something like...
>
> # Creating a parameterless c function.
> compiler.compile( 'printf( "Rock on, dudes!" );'  ).call
>
> # A c function with an integer argument.
> proc = compiler.compile( 'x' => Int, 'printf( "%d * %d = %d", x, x, x*x);' )
> (1..10).each {|x| proc.call(x)}
>
> A module that encapsulated this (particularly as part of the Ruby standard
> distribution) would allow a "pure ruby" module to have native speed [1].
>
> It would also be awesome for dynamic code generation for building an
> algorithm optimised for a very specific situation. Coming from an image
> processing background, I find the idea quite exciting :) As an example, I
> could be using artificial evolution to grow functions that evaluate to a
> pretty picture. To evaluate these functions at maximum speed, I could
> compile them down to c, and execute them.
>
> Another example would be creating a parser from a BNF like notation. It
> should be possible (making use of this module) to specify a grammar with a
> Rubyish domain specific language, and then compile that to native code and
> immediately make use of it from within the same Ruby program.
>
> A further possibility is that it could pull all that pesky linking with
> external libraries (well I don't get on with it anyway, but it's not really
> my bag) in to the domain of Ruby, and make it a fun thing to explore from
> within IRB, for example. An offshoot from that would be an explorative SWIG
> like system: a simple c parser that you can point at a library's header
> file, allowing you to start hacking about with the c library as if you were
> back programming BASIC on your BBC [2] :)
>
> Has anyone come across work in this direction, or is anyone working on it
> themselves? Could anyone comment on difficulties involved, or serious
> practical problems? I think the major issues are likely to be:
>
> 	Interfacing with the native linker.
> 	Passing stuff in and out of the c easily.
> 	Giving the c compiler header files (or bundles, or whatever it needs 
> on OS X).
>
> It's likely I'm completely missing many problems though - I'm coming to this
> having never written a c extension for Ruby!
>
> My dream would be to encapsulate all of the platform specific nitty gritty
> in one place, and so make it reasonably simple for anyone with some c and
> Ruby knowledge to just dive in and start writing super fast code, or linking
> with a library they want to play with.
>
> Sorry for a long post. I hope it will give you ideas, or stimulate interest.
>
> Thank you for your time,
> 	Benjohn Barnes
>
>
> [1] Although one would want a mechanism to be able to cache compiled code 
> between runs eventually, as the compile time could become annoying.
>
> [2] I think Ruby is such an enjoyable experience for me because it recreates
> the fun, and frankly joy, I used to have with BASIC, when I first learnt to
> program, but it allows me to concisely specify my ideas in ways that I've
> come to need after 20 years of programming. What I want to do here is get
> close to having BBC BASIC's build in assembly :) Total expressiveness, and
> raw touching the metal speed and control.

you want to check out rubyinline and ruby/dl.  the later is an interface to
the dynamic linker which allows you load *.so files and call routines in them
directly.  the former spews c code, compiles it, and loads it from pure ruby.
fyi - 'require' __is__ an interface to the dynamic linker, so long as the dll
being loaded is of a certain type.

cheers.

-a

-- 
judge your success by what you had to give up in order to get it.
- h.h. the 14th dali lama