[ruby-core:107420] [Ruby master Bug#18249] The ABI version of dev builds of CRuby does not correspond to the ABI

Issue #18249 has been updated by peterzhu2118 (Peter Zhu).


Hello, I’d like to summarize the motivation for this feature and the implementation I proposed.

# Motivation for this feature

At Shopify, we run Ruby head on CI daily against several services. We also run Ruby head on a small portion of production traffic on some services. This helps catch bugs in Ruby early and makes Ruby upgrades easier. Our services require several hundred gems which take around 30 minutes to install from scratch. To make it faster, we cache these gems. For development Ruby, since the ABI is not stable, cached gems compiled on an older ABI may cause bugs. We can use the ABI as a cache key, and only reinstall gems when the ABI changes. Checking the ABI when loading binaries will give an extra layer of protection to ensure binaries compiled against older header files are not loaded.

We also run daily benchmarks for YJIT at speed.yjit.org. It used to cache gems, but due to the lack of ABI checking in Ruby, yjit-bench must reinstall gems every run to prevent crashes or incorrect performance results due to cached binaries compiled against older header files.

ABI checking is also useful for developers (like myself) who work on Ruby head. I often benchmark my changes against systems like railsbench or discourse. These benchmarks use gems with native extensions. Forgetting to reinstall these gems may cause bugs/crashes or incorrect performance results. ABI checking will ensure that the gems are compiled with the same header files.

# Proposed implementation

It’s important to note that the implementation I proposed will only check ABI for development Ruby. On released Ruby, it does not check the ABI since patch versions of Ruby should ensure ABI compatibility. So this will not affect end users of Ruby, only developers who use Ruby head.

Implementation details can be found in the pull request here: https://github.com/ruby/ruby/pull/5474


----------------------------------------
Bug #18249: The ABI version of dev builds of CRuby does not correspond to the ABI
https://bugs.ruby-lang.org/issues/18249#change-96320

* Author: Eregon (Benoit Daloze)
* Status: Open
* Priority: Normal
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
In fact, it even conflicts with the next release's ABI version:
```
$ ruby -ve 'p RbConfig::CONFIG["ruby_version"]'
ruby 3.1.0dev (2021-10-11T10:13:16Z master 0c3ac87345) [x86_64-linux]
"3.1.0"
```

This mismatch can very easily result in segfaults, memory corruption, etc when using dev versions of CRuby,
or when using a dev version and then later the release.

Possible solutions:
* Include the git commit sha in the ABI. Pros: always correct and simple. Cons: changing more then necessary.
* Track the ABI explicitly, and bump it whenever the ABI changes. Pros: changes only when needed. Cons: easy to forget bumping it, and if checked in CI already too late.

From https://bugs.ruby-lang.org/issues/18239#note-14:

FWIW TruffleRuby actually tracks the ABI of dev versions through [this file](https://github.com/oracle/truffleruby/blob/master/lib/cext/ABI_version.txt) which means it is possible to sensibly cache compiled gems even for dev versions [details](https://github.com/oracle/truffleruby/issues/2284).
Also it [stores](https://github.com/oracle/truffleruby/commit/d497bae73fdc60585b66cd183768a16563b2886b) the ABI version in .so/.bundle files and checks them when loading, so it can be sure the ABI used to compile and runtime ABI match (if not, an exception is raised).

ruby/setup-ruby has no choice for CRuby dev but [to use the commit](https://github.com/ruby/setup-ruby/blob/a6f22865941e122a37e097fbded3dd0b54c39207/bundler.js#L188) as the ABI version.
This issue is made worse by Ruby switchers like RVM & chruby setting GEM_HOME (so the ABI is effectively ignored by RubyGems in those cases, and those directories need to be cleaned manually).
When GEM_HOME is not set, it would be enough to rebuild CRuby dev and remove the directory before installing (which includes both CRuby & gems), but Ruby installers don't do that yet.
Bundler always includes the ABI version when setting the bundler path (`bundle config --local path`), but if the ABI version is incorrect like for CRuby dev it's of no use.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>