[ruby-core:40273] Re: Can rubygems save us from "binary-compatibility hell"?

On Oct 21, 2011, at 7:48 PM, Yusuke Endoh wrote:
> Hello, rubygems developers --
> 
> I'd like to ask some questions about rubygems and Ruby's ABI
> compatibility.
> 
> Currently, the core team tries to ensure ABI compatibility in
> the same Ruby series.  This means an extension library file that
> is compiled with 1.9.1 can be loaded successfully with 1.9.2.
> 
> However, it is sometimes frustrating for the core team to find
> an ABI-compatible way to fix some problems. [ruby-core:28281]
> [ruby-core:27945]
> It is also unfortunate to postpone or give up some new features
> or improvements because they break ABI compatibility.
> [ruby-core:39847]
> In addition, the definition of ABI compatibility is ambiguous
> a little.  I think "ABI compatibility" is actually ensured, but
> some don't. [ruby-core:36108]
> 
> So, it would be good to discard ABI compatibility in 2.0.
> Each version of Ruby (including TEENY) should have their own
> library path, such as:
> 
>  /usr/lib/ruby/2.0.0/
>  /usr/lib/ruby/2.0.1/
>  /usr/lib/ruby/2.0.2/
>  etc.
> 
> In this policy, all the users have to re-install libraries that
> they are using whenever ruby is upgraded.  It's a pain.  But
> recently, many libraries are gems.

Yes, `gem pristine --all` will do this now:

$ GEM_HOME=~/tmp/gems GEM_PATH=~/tmp/gems gem19 pristine --all
Restoring gems to pristine condition...
Building native extensions.  This could take a while...
Restored nokogiri-1.5.0
$

(I used a gem path with just nokogiri to save time)

> So I'd like to ask a question to rubygems developers; is it
> technically possible to automatically re-install all gems from
> old Ruby installation to new one when Ruby is upgraded (or
> during "make install")?

Since 2.0.0 and 2.0.1 will have different gem directories:

$ gem19 env home
/usr/local/lib/ruby/gems/1.9.1

A different mechanism would need to be used to install gems from 2.0.0 into 2.0.1.  Using the cache directory this should be easy without needing to re-download gems, something like:

  cd `gem2.0.0 env home`; gem2.0.1 install --local -f *.gem

But in ruby, of course.

> I have two concerns.  One is a user-custom gem that a user
> created and not published in rubygems.org.  Is there a custom
> gem or its source necessarily remained?

So long as the user has not removed the .gem file from the cache directory (the source) it will work.  Users usually do not know how to do this.

> The other is a binary gem that includes a compiled extension
> library.  TEENY release will be done about every year.  This
> is not a technical issue, but do you think it is reasonable
> to make gem developers release their binary gems when new
> Ruby is released?

I think it depends on the developer.  I can make a list of recent platform gems so we can consult the authors.  With the addition of the Developer Kit for Windows rubyists it may not be as big a concern now as it has been in the past.  Luis can comment better here.

> And, is it possible to allow users to use, with a new Ruby, an old gem that does not support the new Ruby with a warning?

This would require further adjustments to RubyGems, but I think it is possible.  Some of the adjustments are related to the stdlib gems proposal (coming soon).

> And isn't there another concern I missed?
> 
> Note that this is currently just my idea, not a decision at
> all.  What do you think?

I think your idea is possible and easy to implement.