[ruby-core:36308] [Ruby - Bug #921][Open] autoload is not thread-safe

Issue #921 has been updated by Hiroshi NAKAMURA.

Status changed from Closed to Open
Assignee changed from Nobuyoshi Nakada to Shyouhei Urabe
ruby -v changed from - to ruby 1.8.7 (2011-02-18 patchlevel 334) [x86_64-linux]

For JRuby 1.9, I fixed this issue (autoload thread safety.)

Regards to autoload, CRuby 1.9 is thread-safe. CRuby 1.8 is not thread-safe.

  % cat autoload.rb
  class Foo
   autoload :X, 'constant.rb'
  end

  Thread.abort_on_exception = true

  t1 = Thread.new {
    puts "thread #{Thread.current} accessing X"
    p Foo::X
  }
  t2 = Thread.new {
    puts "thread #{Thread.current} accessing X"
    p Foo::X
  }

  t1.join
  t2.join

  % cat constant.rb
  # simulate a slow file load or a deep chain of requires
  puts "#{Thread.current} in constant.rb"

  1_000_000.times { Thread.pass }

  class Foo
    # define X
    X = 1
  end

  % ruby187 autoload.rb
  thread #<Thread:0x7f2e2e301de0> accessing X
  #<Thread:0x7f2e2e301de0> in constant.rb
  thread #<Thread:0x7f2e2e301390> accessing X
  autoload.rb:13: uninitialized constant Foo::X (NameError)
        from autoload.rb:11:in `initialize' 
        from autoload.rb:11:in `new' 
        from autoload.rb:11
  zsh: exit 1     ruby187 autoload.rb

After talking to Shyouhei at Asakusa.rb meetup last night, we agreed that
it's a bug and it should be fixed if we can.

And here's another problem arises. 1.8 removes autoload a constant first as
Charles stated above, then requires the specified file. It's the cause of
threaded autoload issue but it has another side effect. When the require fails
with some Exception, the defined constant is removed afterwards.

  % ruby187 -I. -e 'autoload(:X, "X"); begin; X; rescue LoadError; end; p Object.constants.include?("X")'
  false

1.9 does not remove the constant.

  % ruby -I. -e 'autoload(:X, "X"); begin; X; rescue LoadError; end; p Object.constants.include?(:X)'    
  true

I'm guessing this behavior would be affected when we fix 1.8's thread safety.
Unfortunately, there're "spec"s in RubySpec which expects that constant to be
removed in 1.8.

Can we change this 1.8.7 behavior?
----------------------------------------
Bug #921: autoload is not thread-safe
http://redmine.ruby-lang.org/issues/921

Author: Charles Nutter
Status: Open
Priority: Normal
Assignee: Shyouhei Urabe
Category: 
Target version: 
ruby -v: ruby 1.8.7 (2011-02-18 patchlevel 334) [x86_64-linux]


=begin
 Currently autoload is not safe to use in a multi-threaded application. To put it more bluntly, it's broken.
 
 The current logic for autoload is as follows:
 
 1. A special object is inserted into the target constant table, used as a marker for autoloading
 2. When that constant is looked up, the marker is found and triggers autoloading
 3. The marker is first removed, so the constant now appears to be undefined if retrieved concurrently
 4. The associated autoload resource is required, and presumably redefines the constant in question
 5. The constant lookup, upon completion of autoload, looks up the constant again and either returns its new value or proceeds with normal constant resolution
 
 The problem arises when two or more threads try to access the constant. Because autoload is stateful and unsynchronized, the second thread may encounter the constant table in any number of states:
 
 1. It may see the autoload has not yet fired, if the first thread has encountered the marker but not yet removed it. It would then proceed along the same autoload path, requiring the same file a second time.
 2. It may not find an autoload marker, and assume the constant does not exist.
 3. It may see the eventual constant the autoload was intended to define.
 
 Of these combinations, (3) is obviously the desired behavior. (1) can only happen on native-threaded implementations that do not have a global interpreter lock, since it requires concurrency during autoload's internal logic. (2) can happen on any implementation, since while the required file is processing the original autoload constant appears to be undefined.
 
 I have only come up with two solutions:
 
 * When the autoload marker is encountered, it is replaced (under lock) with an "autoload in progress" marker. All subsequent threads will then see this marker and wait for the autoloading process to complete. the mechanics of this are a little tricky, but it would guarantee concurrent autoloads would only load the target file once and would always return the intended value to concurrent readers.
 * A single autoload mutex, forcing all autoloads to happen in serial.
 
 There is a potential for deadlock in the first solution, unfortunately, since two threads autoloading two constants with circular autoloaded constant dependencies would ultimately deadlock, each waiting for the other to complete. Because of this, a single autoload mutex for all autoloads may be the only safe solution.
=end



-- 
http://redmine.ruby-lang.org