[ruby-core:107483] [Ruby master Feature#18568] Explore lazy RubyGems boot to reduce need for --disable-gems

Issue #18568 has been updated by matthewd (Matthew Draper).


> Over 80% of CRuby's base startup is due to eagerly booting RubyGems. We can do better!

It's not the main point here, but perhaps still worth noting: `--disable-gems` implies `--disable=error_highlight,did_you_mean`.

In my own brief local test, that makes the ratio more like 17% CRuby, 70% rubygems, 13% those two. *

* I do use Gel day-to-day, so I have approximately zero non-default gems installed. It's only two requires, but if you have a lot of gems, that would skew the ratio.

---

Gel does indeed maintain a single index file (well, one per gem installation target, so platform or ruby version for ext gems, and one for all plain-ruby gems). It lost a good portion of its speed when SDBM was kicked out of stdlib, and I had to fall back to PStore, though. (I'm currently experimenting with a load-everything-at-boot strategy, which makes it faster at the cost of more filename strings in memory. Without inventing a new native storage library, I imagine I'll ultimately need to handle custom storage through direct IO, but for now the extra memory is a tolerable trade-off.)

Constructing the index is not overly challenging: during gem installation, you're already responsible for copying the files into their new home, so adding them to an extra list is trivial. (Admittedly, that's less true for default gems.)

----------------------------------------
Feature #18568: Explore lazy RubyGems boot to reduce need for --disable-gems
https://bugs.ruby-lang.org/issues/18568#change-96384

* Author: headius (Charles Nutter)
* Status: Open
* Priority: Normal
----------------------------------------
In https://bugs.ruby-lang.org/issues/17684 there was debate about whether the `--disable-gems` flag should be removed. Several folks were in favor, since Ruby without RubyGems is fairly limited, but others wanted to keep the flag for small, fast command line scripts that do not depend on RubyGems.

Lazily loading RubyGems might be a middle ground, and it has been explored in some depth by TruffleRuby:

https://github.com/oracle/truffleruby/blob/master/src/main/ruby/truffleruby/core/lazy_rubygems.rb

@eregon shows how this improves their startup time in this article from a couple years ago:

https://eregon.me/blog/2019/04/24/how-truffleruby-startup-became-faster-than-mri.html

I believe this approach has merit and could be beneficial to both CRuby and JRuby if we can collaborate on how the lazy loading should happen and figuring out where the edges are. @eregon may know some of those edges if they have run into them in TruffleRuby.

A simple test of `--disable-gems` on CRuby 3.1 shows what an impact it has, which we might be able to duplicate in a lazy boot WITHOUT losing RubyGems functionality and default gem upgrading:

```
$ time ruby -e 1

real    0m0.107s
user    0m0.068s
sys     0m0.030s

$ time ruby --disable-gems -e 1

real    0m0.019s
user    0m0.007s
sys     0m0.008s
```

Over 80% of CRuby's base startup is due to eagerly booting RubyGems. We can do better!



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>