From: Urabe Shyouhei Date: 2011-04-11T13:28:07+09:00 Subject: [ruby-core:35698] Re: mvm branch status? Hi, (04/11/2011 08:00 AM), Charles Oliver Nutter wrote: > On Thu, Apr 7, 2011 at 6:18 PM, Eric Wong wrote: >> Evan Phoenix wrote: >>> Has there been any thought on solving the C extension problem in MVM? >>> In the present state, I've stopped working on it in Rubinius because >>> there is no workable solution if there are C extensions in the mix. >> >> Is it just avoiding redundant load of dynamic libs and remembering to >> call Init_foo() on a per-VM basis? >> >> I suppose the other problem is C-level global variables and >> rb_global_variable(), but not all extensions use those... >> >> I think a useful subset of C extensions can be made to work with MVM, >> but I haven't been thinking of this problem for long (only since reading >> your email :) >> >> Perhaps you or someone can elaborate or point me to a writeup on the >> problem(s). > > Almost all extensions store classes, symbols, constant values, and > more in C globals. If they do that, they can't be made safe with MVM, > since those globals all indirectly reference VM-specific data. And > since we can't determine that they do this on load, ko1 originally > thought to introduce a new Init entry point that's a "promise" by the > extension that it doesn't do MVM-unsafe things. I believe they had a > few extensions working that way when the primary MVM dev push tailed > off. > > Of course we haven't even started to discuss concurrency with C > extensions, which is damn near impossible to do safely. > > In JRuby, where we've had MVM support forever, C extensions will error > out almost immediately if loaded into a second VM. Different VMs can > load different extensions, but the same extension loaded twice will > not work. > > We also have a global lock around extension calls, so only one can be > executing at a given time. > > I strongly believe Ruby needs to have its C API overhauled, or at > least someone needs to seriously explore what APIs can be safe with > MVM and which APIs need fixing/replacing. > > FWIW, none of this applies to JRuby extensions written in Java; they > are bound per VM and do not use any "global" state. > > - Charlie C extension APIs are old (design of 1990s). At least some kind of binary compatibility breakage is unavoidable for Ruby to introduce MVM. On my branch, old style C extensions still work as long as you don't touch any MVM features. But once your ruby script creates a new VM, that VM cannot touch or see those already-required old extensions. To require a C extension form inside that VM, that library should use different APIs (MVM-safe set) as Charles said. I think this is a reasonable trade-off. What C APIs are safe to use with MVM is not fully certain yet. For instance there are process-global states other than C global variables like current directory, signal masks, ... We are still exploring.