Re: Ruby and E.V.E. Paradox

Hey gga,

gga wrote:
 > GD ha escrito:
 >
 >
 >>This is an interesting idea but may be overkill for what I'm trying
 >>to do.
 >
 > Well, I'd suggest you read swig docs and see if you still think is
 > overkill for your project.  Since you mentioned it is a game engine, I
 > seriously doubt that it will be overkill.

I should clarify. My current usage of embedded scripting is fairly
basic. If I went the Ruby route, it would probably grow in scope in
time. Thus my original statement is only half correct. It is overkill
now, it might not be in the future.

I should also emphasise that the game engine handles the bulk of the
game stuff at present. The level scripts are mostly for tutorials or
small exceptions in level behaviour. Thus they are not terribly
demanding.

BUT...

As mentioned I'm weighing up expanding the scope of what the scripts
handle. Small baby steps first...

 >> >   VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );
 >>
 >>You can do this!? Is this safe?
 >
 >
 > Yes.  Again, read ruby.h.  It checks for this to be true... or ruby
 > won't compile.

I remember a mention checking for the size of a number (long?) being
equivalent to a pointer, but not this specifically. Not saying it
isn't there, I just don't remember it.

 >>Is there any authoritative source that
 >>says: "directly casting a pointer to a VALUE is perfectly fine and
 >>will continue to be supported".
 >
 >
 > Besides me, you mean?  I guess we have a trust issue, here :(

Not a trust issue. Just wondering if this is a "well, it works for
me and it's good enough" or "this is the officially sanctioned way
to do it and it'd take a major release before it gets changed".

 > Well, that relies on a feature of the C/C++ language.  As long as you
 > are using C ruby and we are dealing with 32/64-bit machines, it will
 > hold true.  Again, read ruby.h and config.h.
 > Besides that, Matz has mentioned this is ok (search the list).  His
 > goal for VALUE was to have it be like void*, but clearer to users and
 > keeping type safety.  Unfortunately, C does not quite allow it to
 > behave like a void* in all contexts, so sometimes you do need to cast
 > it to void*.
 > This is more or less a limitation of C/C++.
 >
 > If that's not enough for you, I'll try to see if I can convince the
 > Pope to say something along those lines, but it might be tricky :)

I haven't seen it in the list; but I haven't searched it- because
I didn't have any idea it could be the case at all until you
mentioned it. If it does have the Matz-seal-of-approval then that's
good to know- is definitely worth mentioning somewhere in the
official docs as it makes a HUGE difference knowing you can just
cast any pointer to a VALUE rather than having to create an
artificial array to hold it.

And if an official thumbs-up or thumbs-down came along, that would
probably be of enormous benefit to others embedding Ruby.

 >>If so, VALUE can be used (effectively) in the same way as a void*,
 >>so there is no need to convert everything into Ruby objects as
 >>I had thought.
 >
 >
 > Err... you weren't exposing each and every single little class in your
 > code, were you?  Ouch.
 > That's not the idea, of course.  The stuff you need to convert to Ruby
 > is only the stuff you need your users to use in Ruby, as it stands to
 > reason.

No way. No mass class exporting here. I'm actually content with declaring
classes directly in Ruby and doing any needed conversion myself (assuming
I can use opaque data structures too). I do want to expose a fair few
functions/methods from C++ to Ruby though, about twenty or so directly,
and many more indirectly. I need around a half dozen Ruby methods to
be visible to C++ for calling (the event and update methods I mentioned
previously).

 >>I'll probably give it a shot for interest's sake. Looking closer at the
 >>Python docs that don't seem as strong as first impressions suggest,
 >>although there are some decent examples.
 >>
 >
 >
 > Well... what EXACTLY are you trying to achieve, Garret?

Garret is probably a lovely name to have. I also like Garnet as a
name. Unfortunately, around my birth I was named "Garth" and really
haven't seen much reason to change it since. ;)

 > You started
 > the thread mentioning you WANTED to get away from Lua and use Ruby for
 > your game engine.

Yes, that would be great.

This is the progression of thoughts on the issue:

- I was a touch frustrated with LUA, so I thought I'd weigh up replacements.
- After spotting information on Embedded Ruby and skimming through the source,
   I thought it looked promising. Weighing things up, I decided I could get
   additional benefits from scripting with Ruby beyond the current limited
   scripting use. Gave myself two days.
- I gave it a shot. Didn't turn out so well.
- Made a post detailing problems I've been having. Got a bunch of ideas.
   Plan to try many of them out to see which ideas work, and which don't.

The thing of note is that I'm more willing to sink a good chunk of
time into getting Ruby going ahead of other languages, because I *know*
Ruby is a good language to work with. But the time isn't open-ended. If
getting Ruby going would take 2 weeks and Python would take 2 hours,
my new scripting language is Python. If it was 2 days for Ruby and 2
*minutes* for Python, I'd go for Ruby. Basically Ruby gets extra grace
because I more fully understand what it can do.

 > Personally, within the context of game engines, I must tell you, I
 > think that's not such a good idea, as Lua is multi-thread safe today
 > which is definitively something you do want in a game engine (and
 > LuaJIT is probably *THE* fatest VM I have *EVER* seen for a non-static
 > typed language).

The interface to the script will always run in a single thread with
my software, so threading issues aren't a huge concern. Power of
expression is very useful though, hence looking at Ruby.

 > As such, I warned you before-hand that neither Python nor Ruby are
 > thread safe.

Yep.

 > You did mention you liked Ruby's syntax better than anything else and
 > you did not care about multi-threading, so I kept helping you with
 > ruby.

Yes.

 > You mentioned you had difficulties embedding Ruby, and I gave you
 > pointers for you to read on.  You did not understand how to use a
 > couple of functions, and I pointed to you the right stuff you should be
 > using.

Indeed, thanks.

 > You posted some code that crashed on you, and I gave you similar
 > working code that works reliably, with compiling instructions to boot.
 > You questioned whether my code was valid, which means you probably have
 > not tried running it.
 > You have also not followed my advice of looking at SWIG and, instead,
 > now you want to look at Python (!?).  Are you pulling my chain?

I work two jobs. Both involve using Ruby. :) Unfortunately only one
involves Embedded Ruby. Factoring in any time difference, guess which
of the two jobs I've been at recently. ;)

I'll be back and able to do more Embedded Ruby stuff in under 24 hours.
I fully intend to pull in the sample code fragments you've been mentioning
and see how they work with my code. As I mentioned, the SWIG suggestion
is good, and this is also something I will be playing with. Even if I
don't use SWIG, no doubt I can learn much from its output.

I have not questioned your code, just indicated that I haven't been
able to run it yet. My development and net access are on two
completely different non-connected machines, and my present Embedded
Ruby code is a mess- I've been experimenting to find ways around the
current problems I am experiencing. I can't just drop your fixes in,
even if they are 100% perfect; my present Embedded Ruby code is still
a mess!

As for Python, this is what I used quite some time ago. Suddenly one
day a beautiful creature named Ruby came through and swept me off my
feet. Anyway, I don't hate Python, I just like Ruby much more. So
it is still a valid choice for embedding target from my POV. Given
a choice, Ruby is the one for me. But I'm having real trouble with it.

So I'll probably have a play around. I'll see how I go with the new
Ruby knowledge I've picked up here, probably give Python a shot, try
out Scheme, and even give LUA another look. In each case there is
scope for improvement to my code. I do feel the greatest benefit will
come from Ruby though- just a matter of effort versus reward.

I'm basically going through an improvement/experimentation phase with
my software at the moment. Hence my desire to try a few things out.
I'm looking to spend some time now to get payoffs through the next
year.

 > If you DO want to use Python for wrapping a library, and you are
 > familiar with C++ template use, I suggest using boost::python and Py++.
 > It is slightly more efficient than SWIG, albeit it can be hard to
 > debug if you are not familiar with templates.
 >
 > If you don't like templates (like myself), I would ALSO recommend you
 > learn and use SWIG for embedding Python.

Yep, SWIG seems like a good thing to examine regardless.

I didn't know Boost had some Python gear. I'm using various bits of
Boost already. Could be interesting.

 > Is there any reason why you are avoiding SWIG and not even looking at
 > the documentation I pointed you to?

As above, lack of time for the next 24 hours. Then I get a few hours,
sleep, and then most of a day. Trust me, I'm not yet done with this
stuff! I've been given a bunch of useful ideas by various people on
this list (including yourself) and I feel I owe them the courtesy
of trying out some of the suggestions. It all takes time though.

 >  In case you did not find it, it's
 > here, btw:
 > http://www.swig.org/Doc1.3/Ruby.html#Ruby  (read: memory management
 > section)
 > http://www.swig.org/Doc1.3/Contents.html#Contents
 >
 > You will learn quite a lot about embedding stuff from reading it.   My
 > guess is that the problems you are having with Ruby and perhaps Lua
 > might be because you are not too familiar with the subtleties that can
 > arise from exposing classes to scripting languages.  SWIG does a really
 > good job of explaining that (best thing I've seen yet) and has working
 > and relatively well tested code in about 20+ languages.
 >
 > The good thing about SWIG is that you can actually use Lua (which you
 > mentioned you know well already)

My LUA knowledge is "sufficient" only- I wouldn't say I know it well.
I know Ruby well, and Embedded Ruby, barely at all. ;)

 > and expose one or two classes that way
 > first.  Then, you should be able to easily port that code to ruby, by
 > just changing just a couple of swig files (if any).
 > Also, if you end up dropping Ruby at some point in time, you should be
 > able to again port your code easily to whatever other language you
 > fancy.

Bringing SWIG into my project seems an interesting idea. At this point
I'd rather learn from it, but perhaps it would be useful to use the
tool itself directly.

Re the mention of classes, I barely need to share classes at all. As
long as I can pass an opaque pointer to the game class through Ruby
and get it back at the other end, and call functions/methods from
C++ to Ruby to C++, I'm 98% of the way there.

The remaining 2% just involves some initialisation that involves
creating some Ruby objects on the C++ side and having them available
to manipulate with the functions and methods.

 >>(*) - Technically I _did_ in a couple of hours, but it wasn't stable and
 >>crashed randomly, so I tried to figure out what was going wrong. No luck.
 >
 >
 > Well, if you've wrapped a class or two and they are crashing, your best
 > bet is posting the code to them to get help.  Otherwise, it is unlikely
 > anyone will be able to help you out much.

I've got a few snippets I can grab; the main things I'm still
uncertain of are relating to protecting objects etc from the
GC, and how to indicate they are free to clean. I'll post them
as soon as I can.

 > Also, it will be hard for anyone to write better docs, if nobody knows
 > what exactly are you having difficulties with.

Indeed. I've indicated (in length) some of the issues I've been having,
but you are right, I am light on the specifics.

I'd like to minimise the rest of the code as best I can to produce a
bearable test-case that causes problems because as it stands there is
too much non-Embedded-Ruby code for me to sensibly post (and I'm a bit
touchy about some of the non-Ruby code!).

My current code crashes (in the second rb_gc) with something of the form:

void a()
{
   init_ruby();
   rb_gc();
}

void b()
{
   rb_gc();
}

int main(int argc, char **argv)
{
   a();
   b();
}

And starts working when I reorder like this:


void a()
{
   rb_gc();
}

void b()
{
   rb_gc();
}

int main(int argc, char **argv)
{
   init_ruby();
   a();
   b();
}

This triggered my question about whether Ruby makes assumptions about the
stack and if it needs to be called in a persistent frame (not this is not
the case in the first sequence). The actual code of course is much more
complex; we've got calls that cross library boundaries and a few more
stack layers in-between. As mentioned, I'd like to reduce the problem down
to something simpler. But given the simplified code above, could you see
a potential crash from running it like that? If so, that is something
potentially worth mentioning.

Reducing the code down further to a more minimalistic form may expose a
problem on my side as well. If not, I'll have something sensible to
provide as a "is something wrong with this code" example.

 >>I would suggest that a good place for these little tips and tricks that keep 
coming up
 >>would be somewhere near the official documentation, so that the next
 >>person who comes along to play with embedded Ruby can start off
 >>slightly further ahead than I did.
 >>
 >
 >
 > Sure.  Feel free to write them up somewhere.  In case it was not clear,
 > any code or snippet of code I posted in this thread is public-domain.

The best I could manage is to collate your thoughts, as is quite apparent
my Embedded Ruby knowledge is extremely limited. I am completely and
utterly unqualified to write about Embedded Ruby! ;) There are topics
I could write on, but when restricted to Ruby I could at best offer
tutorials to new users. The existing ones are already pretty good.

Now it's 11:30pm, so I'm off to bed. Hopefully I'll be back to embedded
language experimentation next evening after my other job.

Garth