From: "Eregon (Benoit Daloze)" Date: 2022-02-03T18:46:35+00:00 Subject: [ruby-core:107474] [Ruby master Feature#18568] Explore lazy RubyGems boot to reduce need for --disable-gems Issue #18568 has been updated by Eregon (Benoit Daloze). > https://github.com/rubygems/rubygems/pull/4199 Interesting, that seems a helpful step, although I suspect there would still be a significant overhead over `--disable-gems`, as long as we load so much Ruby code during startup. I think `upgraded_default_gem?` in https://github.com/oracle/truffleruby/blob/14ec2c2673188d47374a0570cf036864fcafe0b3/src/main/ruby/truffleruby/core/truffle/gem_util.rb#L87 has two advantages compared to that approach: * It hardcodes the list of default gem names, which AFAIK is anyway not possible to change for a given Ruby release, but iterating the default directory for default gem names would likely be fast enough. * It assumes default gems have sensible require-able files, so `require 'foo/bar/...'` if it's a default gem file must be for default gem `foo`, which holds for all current default gems AFAIK. Just the special case of gem names with a `-` in it like `net/http`, then we it checks if the path starts with `net` (there is anyway no stdlib starting with `net` or `io` which is not a default gem nowadays). That avoids e.g. having a list of all files which is large memory footprint and rather time consuming to get. The purpose of `upgraded_default_gem?` is to avoid loading RubyGems when loading non-gem stdlib files. --- BTW I think such an optimization would be very useful for non-default gems too. Right now `require 'not-a-default-gem'` or `require 'not-a-default-gem/foo.rb'` loads all gemspecs (so it's `O(installed gems)`, which is problematic with many gems installed), while it very likely is in a gem named `not-a-default-gem`. This can affect semantics for gems which don't respect the convention and might be earlier alphabetically than a gem with the proper name, not sure that matters though. ---------------------------------------- Feature #18568: Explore lazy RubyGems boot to reduce need for --disable-gems https://bugs.ruby-lang.org/issues/18568#change-96377 * Author: headius (Charles Nutter) * Status: Open * Priority: Normal ---------------------------------------- In https://bugs.ruby-lang.org/issues/17684 there was debate about whether the `--disable-gems` flag should be removed. Several folks were in favor, since Ruby without RubyGems is fairly limited, but others wanted to keep the flag for small, fast command line scripts that do not depend on RubyGems. Lazily loading RubyGems might be a middle ground, and it has been explored in some depth by TruffleRuby: https://github.com/oracle/truffleruby/blob/master/src/main/ruby/truffleruby/core/lazy_rubygems.rb @eregon shows how this improves their startup time in this article from a couple years ago: https://eregon.me/blog/2019/04/24/how-truffleruby-startup-became-faster-than-mri.html I believe this approach has merit and could be beneficial to both CRuby and JRuby if we can collaborate on how the lazy loading should happen and figuring out where the edges are. @eregon may know some of those edges if they have run into them in TruffleRuby. A simple test of `--disable-gems` on CRuby 3.1 shows what an impact it has, which we might be able to duplicate in a lazy boot WITHOUT losing RubyGems functionality and default gem upgrading: ``` $ time ruby -e 1 real 0m0.107s user 0m0.068s sys 0m0.030s $ time ruby --disable-gems -e 1 real 0m0.019s user 0m0.007s sys 0m0.008s ``` Over 80% of CRuby's base startup is due to eagerly booting RubyGems. We can do better! -- https://bugs.ruby-lang.org/ Unsubscribe: