[ruby-core:35698] Re: mvm branch status?

Hi,

(04/11/2011 08:00 AM), Charles Oliver Nutter wrote:
> On Thu, Apr 7, 2011 at 6:18 PM, Eric Wong <[e-mail]> wrote:
>> Evan Phoenix <[e-mail]> wrote:
>>> Has there been any thought on solving the C extension problem in MVM?
>>> In the present state, I've stopped working on it in Rubinius because
>>> there is no workable solution if there are C extensions in the mix.
>>
>> Is it just avoiding redundant load of dynamic libs and remembering to
>> call Init_foo() on a per-VM basis?
>>
>> I suppose the other problem is C-level global variables and
>> rb_global_variable(), but not all extensions use those...
>>
>> I think a useful subset of C extensions can be made to work with MVM,
>> but I haven't been thinking of this problem for long (only since reading
>> your email :)
>>
>> Perhaps you or someone can elaborate or point me to a writeup on the
>> problem(s).
> 
> Almost all extensions store classes, symbols, constant values, and
> more in C globals. If they do that, they can't be made safe with MVM,
> since those globals all indirectly reference VM-specific data. And
> since we can't determine that they do this on load, ko1 originally
> thought to introduce a new Init entry point that's a "promise" by the
> extension that it doesn't do MVM-unsafe things. I believe they had a
> few extensions working that way when the primary MVM dev push tailed
> off.
> 
> Of course we haven't even started to discuss concurrency with C
> extensions, which is damn near impossible to do safely.
> 
> In JRuby, where we've had MVM support forever, C extensions will error
> out almost immediately if loaded into a second VM. Different VMs can
> load different extensions, but the same extension loaded twice will
> not work.
> 
> We also have a global lock around extension calls, so only one can be
> executing at a given time.
> 
> I strongly believe Ruby needs to have its C API overhauled, or at
> least someone needs to seriously explore what APIs can be safe with
> MVM and which APIs need fixing/replacing.
> 
> FWIW, none of this applies to JRuby extensions written in Java; they
> are bound per VM and do not use any "global" state.
> 
> - Charlie

C extension APIs are old (design of 1990s).  At least some kind of binary
compatibility breakage is unavoidable for Ruby to introduce MVM.  On my
branch, old style C extensions still work as long as you don't touch any MVM
features.  But once your ruby script creates a new VM, that VM cannot touch or
see those already-required old extensions.  To require a C extension form
inside that VM, that library should use different APIs (MVM-safe set) as
Charles said.  I think this is a reasonable trade-off.

What C APIs are safe to use with MVM is not fully certain yet.  For instance
there are process-global states other than C global variables like current
directory, signal masks, ... We are still exploring.