From: "fxn (Xavier Noria)" Date: 2022-10-04T13:46:36+00:00 Subject: [ruby-core:110180] [Ruby master Feature#19024] Proposal: Import Modules Issue #19024 has been updated by fxn (Xavier Noria). Just a very quick followup: > But I'd like to separate it from the question of whether the idea is even practically possible. Oh, if the purpose of this exploration is to see if doing this is technically doable then by all means go ahead :). > Putting my cards on the table, I personally have always found the assumption that all Ruby code is loaded from toplevel to be one of Ruby's biggest weaknesses. That's my view, and I'm happy to elaborate on it, but my focus right now will be objectively on whether this toplevel-centric design is inevitable or not. I believe your pain point is that Ruby does not have formal namespaces, and we share it. The emulation via class/module objects and constants is a bit weak. Please, note that other languages like Java always work in the global namespace (Java does not need imports like Python does). But the strict file structure and the convention of using domain names as package names resuts in separation in practice. I'll ponder the rest of the reply later, also looking forward to that example! :) ---------------------------------------- Feature #19024: Proposal: Import Modules https://bugs.ruby-lang.org/issues/19024#change-99455 * Author: shioyama (Chris Salzberg) * Status: Open * Priority: Normal ---------------------------------------- There is no general way in Ruby to load code outside of the globally-shared namespace. This makes it hard to isolate components of an application from each other and from the application itself, leading to complicated relationships that can become intractable as applications grow in size. The growing popularity of a gem like [Packwerk](https://github.com/shopify/packwerk), which provides a new concept of "package" to enfoce boundaries statically in CI, is evidence that this is a real problem. But introducing a new packaging concept and CI step is at best only a partial solution, with downsides: it adds complexity and cognitive overhead that wouldn't be necessary if Ruby provided better packaging itself (as Matz has suggested [it should](https://youtu.be/Dp12a3KGNFw?t=2956)). There is _one_ limited way in Ruby currently to load code without polluting the global namespace: `load` with the `wrap` parameter, which as of https://bugs.ruby-lang.org/issues/6210 can now be a module. However, this option does not apply transitively to `require` calls within the loaded file, so its usefulness is limited. My proposal here is to enable module imports by doing the following: 1. apply the `wrap` module namespace transitively to `require`s inside the loaded code, including native extensions (or provide a new flag or method that would do this), 2. make the `wrap` module the toplevel context for code loaded under it, so `::Foo` resolves to `::Foo` in loaded code (or, again, provide a new flag or method that would do this). _Also make this apply when code under the wrapper module is called outside of the load process (when `top_wrapper` is no longer set) — this may be quite hard to do_. 3. resolve `name` on anonymous modules under the wrapped module to their names without the top wrapper module, so `::Foo.name` evaluates to `"Foo"`. There may be other ways to handle this problem, but a gem like Rails uses `name` to resolve filenames and fails when anonymous modules return something like `#::ActiveRecord` instead of just `ActiveRecord`. I have roughly implemented these three things in [this patch](https://github.com/ruby/ruby/compare/master...shioyama:ruby:import_modules). This implementation is incomplete (it does not cover the last highlighted part of 2) but provides enough of a basis to implement an `import` method, which I have done in a gem called [Im](https://github.com/shioyama/im). Im provides an `import` method which can be used to import gem code under a namespace: ```ruby require "im" extend Im active_model = import "active_model" #=> <#Im::Import root: active_model> ActiveModel #=> NameError active_model::ActiveModel #=> ActiveModel active_record = import "active_record" #=> <#Im::Import root: active_record> # Constants defined in the same file under different imports point to the same objects active_record::ActiveModel == active_model::ActiveModel #=> true ``` With the constants all loaded under an anonymous namespace, any code importing the gem can name constants however it likes: ```ruby class Post < active_record::ActiveRecord::Base end AR = active_record::ActiveRecord Post.superclass #=> AR::Base ``` Note that this enables the importer to completely determine the naming for every constant it imports. So gems can opt to hide their dependencies by "anchoring" them inside their own namespace, like this: ```ruby # in lib/my_gem.rb module MyGem dep = import "my_gem_dependency" # my_gem_dependency is "anchored" under the MyGem namespace, so not exposed to users # of the gem unless they also require it. MyGemDependency = dep #... end ``` There are a couple important implementation decisions in the gem: 1. _Only load code once._ When the same file is imported again (either directly or transitively), "copy" constants from previously imported namespace to the new namespace using a registry which maps which namespace (import) was used to load which file (as shown above with activerecord/activemodel). This is necessary to ensure that different imports can "see" shared files. A similar registry is used to track autoloads so that they work correctly when used from imported code. 2. Toplevel core types (`NilClass`, `TrueClass`, `FalseClass`, `String`, etc) are "aliased" to constants under each import module to make them available. Thus there can be side-effects of importing code, but this allows a gem like Rails to monkeypatch core classes which it needs to do for it to work. 3. `Object.const_missing` is patched to check the caller location and resolve to the constant defined under an import, if there is an import defined for that file. To be clear: **I think 1) should be implemented in Ruby, but not 2) and 3).** The last one (`Object.const_missing`) is a hack to support the case where a toplevel constant is referenced from a method called in imported code (at which point the `top_wrapper` is not active.) I know this is a big proposal, and there are strong opinions held. I would really appreciate constructive feedback on this general idea. See also similar discussion in: https://bugs.ruby-lang.org/issues/10320 -- https://bugs.ruby-lang.org/ Unsubscribe: