From: Charlie Savage Date: 2005-08-19T08:53:12+09:00 Subject: SWIG Ruby Memory Management Hi everyone, I've recently gotten up to speed with SWIG as I try to create Ruby bindings for the GDAL open source project. SWIG Ruby supports seems pretty good with one exception - memory management. I have run into three issues: * There is no support for telling SWIG that ownership of an object has changed * There is no built in infrastructure for making it easy to implement a mark function * Multiple Ruby objects can wrap the same C++ object / C struct. I describe these issues in more detail in these two posts: http://mailman.cs.uchicago.edu/pipermail/swig-dev/2005-August/014991.html http://mailman.cs.uchicago.edu/pipermail/swig-dev/2005-August/014992.html Since these seem to be problems that many people run into when writing Ruby extensions with SWIG, I have spent some time implementing generic solutions for them and would like to run them by the list for comment/feedback/suggestions. API --------- The basic api is: /* Add a mapping from a C/C++ struct to a Ruby object */ static void SWIG_RubyAddMapping(void* ptr, VALUE object); /* Get the Ruby object that owns the specified C/C++ struct */ static VALUE SWIG_RubyInstanceFor(void* ptr); /* Remove a mapping from a C/C++ struct to a Ruby object */ static void SWIG_RubyRemoveMapping(void* ptr); The SWIG wrapper code takes care of calling RubyAddMapping and RubyRemoveMapping when Ruby objects in extensions are created/deleted. Thus the extension programmer generally only has to call SWIG_RubyInstanceFor when implementing a mark function (although in some cases you may have to call SWIG_RubyRemoveMapping if your are manually destroying objects on the C++ side). Implementation ---------------- To solve issues #2 and #3 requires keeping track of mappings between C/C++ structs and Ruby objects. Reading through various newsgroups I have seen two general approaches - have a hash table that stores these mappings or add an field to each C/C++ struct to store a reference to a Ruby object. As can be seen in the code below, I have gone down the hashtable route with a slight twist - I use a Ruby hash table from C to store these mappings. I did this instead of using a C++ map since I wanted to the code to be just C and it was easier then implementing a hash table from scratch in C. Performance --------------- To make sure using a Ruby hash table performed adequately, I ran some performance tests. Test - Create 100,000 Animal (implemented in an extension, taken from the SWIG Ruby manual). The test machine was a Pentium M Thinkpad T41 laptop. Test Time No mappings 9.3 to 9.6 seconds Mappings using Ruby hash 9.5 to 9.9 seconds Mappings using std::map 30 seconds You can see there is slight performance hit for tracking mappings between C++ objects and Ruby object. Note the C++ map was so slow because deletions took a lot of time (and there were 100,000 deletetions by the end of the test run). Summary --------- If people think it would be a good idea to add this functionality to SWIG (whether using this implementation or some other one) then I'd be glad to take the lead on this. Thanks, Charlie Code --------------------- This is a new file called rubymappings.swg /* Global Ruby hash table to store mappings from C/C++ structs to Ruby Objects. */ static VALUE swig_ruby_mappings; /* Get a Ruby number to reference a pointer */ static VALUE SWIG_RubyPtrToReference(void* ptr) { /* We cast the pointer to an unsigned long and then store a reference to it using a Ruby number object. */ /* Convert the pointer to a Ruby number */ unsigned long value = (unsigned long) ptr; return LONG2NUM(value); } /* Get a Ruby number to reference an object */ static VALUE SWIG_RubyObjectToReference(VALUE object) { /* We cast the object to an unsigned long and then store a reference to it using a Ruby number object. */ /* Convert the Object to a Ruby number */ unsigned long value = (unsigned long) object; return LONG2NUM(value); } /* Get a Ruby object from a previously stored reference */ static VALUE SWIG_RubyReferenceToObject(VALUE reference) { /* The provided Ruby number object is a reference to the Ruby object we want.*/ /* First convert the Ruby number to a C number */ unsigned long value = NUM2LONG(reference); return (VALUE) value; } /* Add a mapping from a C/C++ struct to a Ruby object */ static void SWIG_RubyAddMapping(void* ptr, VALUE object) { /* In a Ruby hash table we store the pointer and the associated Ruby object. The trick here is that we cannot store the Ruby object directly - if we do then it cannot be garbage collected. So instead we typecast it as a unsigned long and convert it to a Ruby number object.*/ /* Get a reference to the pointer as a Ruby number */ VALUE key = SWIG_RubyPtrToReference(ptr); /* Get a reference to the Ruby object as a Ruby number */ VALUE value = SWIG_RubyObjectToReference(object); rb_hash_aset(swig_ruby_mappings, key, value); } /* Get the Ruby object that owns the specfied C/C++ struct */ static VALUE SWIG_RubyInstanceFor(void* ptr) { /* Get a reference to the pointer as a Ruby number */ VALUE key = SWIG_RubyPtrToReference(ptr); /* Now lookup the value stored in the Ruby hash table */ VALUE value = rb_hash_aref(swig_ruby_mappings, key); if (value == Qnil) { return Qnil; } else { /* Convert this value to Ruby object */ return SWIG_RubyReferenceToObject(value); } } /* Remove a mapping from a C/C++ struct to a Ruby object */ static void SWIG_RubyRemoveMapping(void* ptr) { /* Get a reference to the pointer as a Ruby number */ VALUE key = SWIG_RubyPtrToReference(ptr); VALUE object = SWIG_RubyInstanceFor(ptr); /* Reset the C/C++ data struct associated with the Object. This is needed in case a Ruby object exists longer than its underlying C++ object. By setting the data_struct to nil, code in SWIG_Ruby_ConvertPtr can detect this problem and return an error message, as opposed to causing a segmentation fault.*/ if (object != Qnil) { DATA_PTR(object) = 0; } /* Now delete the object from the hash table. To do this we need to call the Hash.delete method in Ruby. */ static VALUE delete_function = rb_intern("delete"); rb_funcall(swig_ruby_mappings, delete_function, 1, key); } /* Setup a Ruby hash table to store mappings */ void SWIG_RubyInitializeMappings() { /* Create a ruby hash table to store mappings from C++ objects to Ruby objects. Also make sure to tell the garabage collector about the hash table */ swig_ruby_mappings = rb_hash_new(); rb_gc_register_address(&swig_ruby_mappings); } Patches to Hook Into SWIG ---------------------- Index: ruby.cxx =================================================================== RCS file: /cvsroot/swig/SWIG/Source/Modules/ruby.cxx,v retrieving revision 1.78 diff -u -i -r1.78 ruby.cxx --- ruby.cxx 17 Aug 2005 20:35:53 -0000 1.78 +++ ruby.cxx 18 Aug 2005 18:41:22 -0000 @@ -465,6 +465,9 @@ NIL); Printf(f_init,"\n"); + // Initialize mapping code + Printf(f_init,"SWIG_RubyInitializeMappings();\n"); + Language::top(n); /* Finish off our init function */ @@ -1098,7 +1107,8 @@ if (current == CONSTRUCTOR_INITIALIZE) { String *action = Getattr(n,"wrap:action"); if (action) { - Append(action,"DATA_PTR(self) = result;"); + Append(action,"DATA_PTR(self) = result;\n"); + Append(action,"SWIG_RubyAddMapping(result, self);\n"); } } emit_action(n,f); @@ -1850,6 +1860,8 @@ Printf(freebody, "free((char*) %s);\n", Swig_cparm_name(0,0)); } } + Printf(freebody, tab4, NIL); + Printf(freebody, "SWIG_RubyRemoveMapping(%s);\n", Swig_cparm_name(0,0)); Printv(freebody, "}\n", NIL); Printv(f_wrappers, freebody, NIL); Index: ruby.swg =================================================================== RCS file: /cvsroot/swig/SWIG/Lib/ruby/ruby.swg,v retrieving revision 1.29 diff -u -i -r1.29 ruby.swg --- ruby.swg 1 Feb 2005 00:08:19 -0000 1.29 +++ ruby.swg 18 Aug 2005 18:44:19 -0000 @@ -6,6 +6,7 @@ %runtime "rubyhead.swg" %runtime "swigrun.swg" // Common C API type-checking code +%runtime "rubymappings.swg" /* Stores mappings from C/C++ struct to Ruby objects */ %runtime "rubydef.swg" %insert(initbeforefunc) "swiginit.swg" Index: rubydef.swg =================================================================== RCS file: /cvsroot/swig/SWIG/Lib/ruby/rubydef.swg,v retrieving revision 1.17 diff -u -i -r1.17 rubydef.swg --- rubydef.swg 1 Feb 2005 00:08:20 -0000 1.17 +++ rubydef.swg 18 Aug 2005 18:43:53 -0000 @@ -71,6 +71,13 @@ if (!ptr) return Qnil; + + /* Have we already wrapped this pointer? */ + obj = SWIG_RubyInstanceFor(ptr); + + if (obj != Qnil) { + return obj; + } if (type->clientdata) { sklass = (swig_class *) type->clientdata;