From: "fxn (Xavier Noria) via ruby-core" Date: 2025-02-12T14:17:29+00:00 Subject: [ruby-core:120955] [Ruby master Feature#15663] Documenting autoload semantics Issue #15663 has been updated by fxn (Xavier Noria). I see what you mean, but let me rephrase it a bit. The autoload does not depend on the nesting (depending on your definition of "autoload"). It is triggered by a missing constant reference when the receiver is the cref, and the execution of the required file is not influence by the nesting in the caller (you of course know this, only describing). Success/failure of the `require` call does not depend on the nesting. Constant lookup does depend on the nesting, but constant lookup and autoload are independent of each other (except that one may trigger the other one). I agree with you in those rules, I would also prefer this to be simpler and more strict. Either you define the constant in the receiver or error. I'd go further for the sake of the exercise (but impossible to change). An autoload is a promise, but it may fail. In that sense, if this was day 0 and I was the author of the language, autoloads would not appear in the constants API. Only "actual" constants would. ---------------------------------------- Feature #15663: Documenting autoload semantics https://bugs.ruby-lang.org/issues/15663#change-111850 * Author: Eregon (Benoit Daloze) * Status: Open ---------------------------------------- The semantics of autoload are extremely complicated. As far as I can see, they are unfortunately not documented. ruby/spec tries to test [many aspects](https://github.com/ruby/spec/blob/master/core/module/autoload_spec.rb) of it, `test/ruby/test_autoload.rb` has a few tests, and e.g. zeitwerk tests [some other parts](https://github.com/fxn/zeitwerk/blob/master/test/lib/zeitwerk/test_ruby_compatibility.rb). One could of course read the MRI source code, but I find it very hard to follow around `autoload`. For the context, I'm trying to implement `autoload` as correct as possible in TruffleRuby and finding it very difficult given the inconsistencies (see below) and lack of documentation. There is nowhere a document on how it should behave, and given the complexity of it I am not even sure MRI behaves as expected. Could we create this document? For instance, there is such a [document for refinements](https://github.com/ruby/ruby/blob/trunk/doc/syntax/refinements.rdoc). Here is an example how confusing autoload can be, and I would love to hear the rationale or have some written semantics on why it is that way. main.rb: ```ruby require "pp" $: << __dir__ Object.autoload(:Foo, "foo") CHECK = -> state { checks = -> { { defined: defined?(Foo), const_defined: Object.const_defined?(:Foo), autoload?: Object.autoload?(:Foo), in_constants: Object.constants.include?(:Foo), } } pp when: state, **checks.call, other_thread: Thread.new { checks.call }.value } CHECK.call(:before_require) if ARGV.first == "require" require "foo" else Foo # trigger the autoload end CHECK.call(:after) p Foo ``` foo.rb: ```ruby CHECK.call(:during_before_defining) module Foo end CHECK.call(:during_after_defining) ``` Here are the results for MRI 2.6.1: ```ruby $ ruby main.rb {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, the constant looks not defined during the autoload for the Thread loading it, but looks defined and as an autoload for other threads. Now we can discover other subtle semantics, by using `require` on the autoload file instead of accessing the constant: ```ruby $ ruby main.rb require {:when=>:before_require, :defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>"foo", :in_constants=>true}} {:when=>:during_before_defining, :defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>nil, :const_defined=>false, :autoload?=>nil, :in_constants=>true}} {:when=>:during_after_defining, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} {:when=>:after, :defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true, :other_thread=> {:defined=>"constant", :const_defined=>true, :autoload?=>nil, :in_constants=>true}} Foo ``` Looking at during_before_defining, now the other threads seem to see the constant not defined, although it is still in `Object.constants`. But of course, the constant cannot be removed, as otherwise that would not be thread-safe and other threads would raise NameError when accessing the constant. In fact, we can see other threads actually wait for the constant, by changing to `Thread.new { Foo; checks.call }`, and then we get a deadlock: ``` Traceback (most recent call last): 2: from main.rb:20:in `
' 1: from main.rb:17:in `block in
' main.rb:17:in `value': No live threads left. Deadlock? (fatal) 3 threads, 3 sleeps current:0x00007f0124004cb0 main thread:0x000055929cc2c470 * # rb_thread_t:0x000055929cc2c470 native:0x00007f013381d700 int:0 main.rb:17:in `value' main.rb:17:in `block in
' main.rb:20:in `
' * # rb_thread_t:0x000055929ce026d0 native:0x00007f0129007700 int:0 depended by: tb_thread_id:0x000055929cc2c470 main.rb:17:in `value' main.rb:17:in `block in
' foo.rb:1:in `' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' /home/eregon/.rubies/ruby-2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require' main.rb:17:in `block (2 levels) in
' * # rb_thread_t:0x00007f0124004cb0 native:0x00007f0128e05700 int:0 depended by: tb_thread_id:0x000055929ce026d0 main.rb:17:in `block (2 levels) in
' ``` This is quite weird. Is the second behavior a bug? Why should other threads suddenly see the constant as "not defined" while it is loading via `require` in the main thread? It's also inconsistent with the first case. I would have thought `require autoload_path` would basically do the same as triggering the autoload of the constant (such as `Foo`). But the results above show they differ. There are many more complex cases for autoload, such as [this spec](https://github.com/ruby/spec/blob/72bd058b5cf0a9d9de5a188052db2fba021581cc/core/module/autoload_spec.rb#L360-L375), or how is thread-safety is achieved when methods are defined incrementally in Ruby but the module is defined immediately. Who is knowledgeable about `autoload` and could answer these questions? Could we start a document specifying the semantics? -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/