Ruby and E.V.E. Paradox

I have been using Ruby for several years. Most importantly I have been using 
this amazing language as a support language for E.V.E. Paradox, a game suite I 
have been working on for much of the last three years. Ruby is used extensively 
in the tools used to build and test E.V.E. Paradox. It has literally saved me 
hundreds of hours. I use Ruby to generate many thousands of lines of C++ source 
from configuration files, automate builds, automatically fix problems with other 
software, generate dynamic web content, manage data, test, maintain the 
codebase, and so forth. It is reliable, easy to use, intuitive, and a fantastic 
base for rapid development of tools. To everyone involved in the development of 
this amazing language, thankyou.

Recently I have upgraded my development machine and with it, all of the support 
libraries I use with my software. I presently use LUA for embedded scripting in 
E.V.E. Paradox. After the upgrade I found that several key API calls had been 
renamed (for no reason AFAICT), and the existing scripts, even when fixed, no 
longer works. I have found LUA generally difficult to work with, and only use 
the basics of the language anyway. The documentation is less than ideal. I 
started entertaining the idea that it would be very, *very* nice if I could use 
Ruby as my scripting language instead; basically Ruby is solid and powerful, and 
I have years of experience with it.

This is what I needed to replace LUA:
- Ability to load a script *from*memory*, not from a file (important).
- Run any init code in the script.
- Hook a bunch of local functions (C++) so that the script can call them for 
information from the main code body.
- Call functions in the script both periodically and in response to events (eg. 
an update call every 1/60 of a second, in response to mouse clicks, etc).
- Clean up everything entirely when the level is over.
- Be prepared to run everything again later when the next level needing 
scripting is loaded.
- Ideally have separate contexts so that two separate games running at once (in 
the same exe) can work in their own scripting environment.

I noticed that Ruby is built with many functions available in a separate dynamic 
library, which builds easily both under Linux and through MinGW, and has some 
kind of embeddable interface. I figured there was a chance I could get things 
going, so I gave myself two days to figure things out and get the basics running.

The process has been a complete nightmare. I have been plagued by constant 
crashes and uncertain documentation. I have been at this for close to two full 
days. I am using ruby-1.8.5-p12 built from source, LD_LIBRARY_PATH is fine, and 
other instances of Ruby have been completely removed from the machine.

This is what I have encountered:

- There doesn't appear to be any way to indicate that Ruby should clean 
everything up and get to a state where I can start again. I have attempted to 
work around this by making each script contain a single unique class that I use, 
but this is a terrible hack.
- How do I load a script from memory? rb_eval_string()? rb_compile_string()? 
Something else?
- The documentation is flawed. It indicates calling rb_run(), but that can 
eventually call exit, which is the last thing you want from an embedded 
language. It mentions rb_apply() as having an argc parameter.
- I am getting inexplicable crashes when calling the garbage collector. Fearing 
it was my mistake, I eliminated calls repeatedly until I got down to ruby_init() 
and two calls to rb_gc(). This crashed. I have searched around and noticed 
people with comparable problems, and did the same thing. I have seen 
condescending replies with no indication as to why a simple code sequence such 
as this could cause a crash. Not calling the garbage collector is NOT a 
solution, as it could be called outside of my control and I don't want my code 
randomly crashing when it does.
- I managed to mitigate some crashes by moving the Ruby initialisation from 
inside a function to near the code initialisation such that the stack present on 
creation will be in place by the time other calls are made. This made a 
difference! Is the ruby interpreter performing some operations on the real stack 
that I don't know about? Shouldn't something like this be mentioned somewhere?-
- I wrapped rb_gc() in a rb_protect() call, didn't help.
- The documentation is not at all clear how you protect something from the 
garbage collector, or indicate it is no longer in use.
- How does one call "Foo.new(args)" on a class from outside of Ruby, where the 
args have been generated from outside of Ruby (hence no name) meaning 
rv_eval_string is unavailable, and I'm not sure how rb_funcall()/rb_apply() can 
be used to call "new" on a class to get an object?
- rb_protect() is a complete mess. Rather than being able to call an arbitrary 
function with a void pointer, you are stuck calling something that takes a 
single VALUE arg. This means that if you want to call a general function, you 
need to resort to tricks like building up a ruby array with the receiver, method 
string, and real args (also in their own array), pass it through, and 
deconstruct it on the other end, feeding it into rb_apply() rather than using 
the more convenient rb_funcall() calls. Writing multiple dedicated callbacks is 
not an option, as I lose the local context (eg. "this") in C++ when I do that, 
unless I go to the extra effort to wrap it. Is there some reason it is like this?
- The code runs fine for a while if I don't call the garbage collector. 
Unfortunately, I get stack depth errors and eventual segfaults if I make 
repeated calls to the Ruby script (via rb_apply()). Since I need to call an 
"update" function sixty times a second, minimising my call use is not an option.
- When searching for errors that come up during development (eg. stack size 
problems, crashes on garbage collection) I get a huge number of hits. I am not 
alone here, and it seems people are having genuine trouble finding a solution.
- I can't find one example that completely and correctly demonstrates the whole 
process involved in having C call Ruby then call C, return values through the 
chain neatly, and clean up. If there was such a reference, it would be outstanding.
- There seem to be external efforts to clean things up, but they aren't making 
it back into the official code. Is the embedded interface not really supposed to 
be used for anything but extensions to Ruby?

I wonder if Ruby is at all suitable as a general-purpose embedded scripting 
language? I am beginning to think that it is not.

The embedded interface and documentation are terrible! I don't mean any offense, 
and I maintain my stance that Ruby is an outstanding language. I love it. And I 
do realise that sometimes code takes some time to mature- I've been using Ruby 
since the 90's and it wasn't always a smooth ride. Is this the problem, it just 
isn't quite ready for embedding yet? Or I'm not using the right documentation? 
Or I'm asking too much? Am I doing the wrong thing?

It probably seems odd for someone with such a high opinion of the language to 
have such a low opinion of the embedded interface. Perhaps it is because Ruby 
has set my expectations so high that I am that much more easily disappointed.

Some projects seem to have managed it, and they may work, to some degree. But 
their use of it seems to be fairly basic, rather than counting on it as a core 
component of the program. VIM has a nice example, although it doesn't seem to 
worry about garbage collection or cleaning up. As such it doesn't seem complete.

Am I alone in thinking this? Should I be looking at other embeddable languages 
(Python, for example), or just go back and get the LUA code working again? Am I 
doing something wrong?

Is there anything I can do to successfully and reliably add support for my 
favourite scripting language into my project? Or is it simply an unrealistic 
prospect?

Garth