From: naruse@... Date: 2015-10-27T03:07:28+00:00 Subject: [ruby-core:71204] [Ruby trunk - Bug #11572] [Closed] Urnary operator causing references to unreachable objects in 2.1.x ? Issue #11572 has been updated by Yui NARUSE. Tracker changed from Backport to Bug Project changed from Backport21 to Ruby trunk Status changed from Open to Closed ruby -v set to ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux] Backport set to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONTNEED ---------------------------------------- Bug #11572: Urnary operator causing references to unreachable objects in 2.1.x ? https://bugs.ruby-lang.org/issues/11572#change-54583 * Author: Danny Guinther * Status: Closed * Priority: Normal * Assignee: * ruby -v: ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux] * Backport: 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONTNEED ---------------------------------------- Perhaps this is an error on my part, but I stumbled across some weird GC behavior related to the unary & (ampersand) operator on 2.1.x. I don't have any leads as to what the cause of the issue might be, but the gist of the issue is that using & with Array#each or Array#map seems to cause references to unreachable objects to be maintained, preventing those unreferenced objects from being GC'd. The majority of my testing has been on Ubuntu 14.04.3, though a colleague was kind enough to verify that the behavior also occurs on OSX. This seems like it is likely related to https://github.com/ruby/ruby/pull/592 which was ultimately solved by commit 2f3b28c682fe3010ed3b8803199616c12b52512d: +Sat Apr 12 22:11:10 2014 Nobuyoshi Nakada + + * string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear + caller's self which is useless, so that it can get collected. + [Fixes GH-592] As far as I can tell, this commit was not backported to 2.1.x. If this commit did fix the issue, should it be backported to 2.1? I haven't seen been able to find an existing bug for this issue if one exists, so it's unclear to me why this wouldn't have been backported. I've been using the script below to experiment with the phenomenon. I also made a gist of the script here: https://gist.github.com/tdg5/0b9f145edb5114a2dca1 ~~~ # Create some special classes to facilitate tracking allocated objects. class TrackedArray < Array; end class TrackedString < String; end STRANG = "a" * 5000 class ClingyObjects def generate(should_cling = false) strs = TrackedArray.new 30000.times { strs << TrackedString.new(STRANG) } char_count = 0 # I'm not sure why, but using the unary & operator on the Array, whether # through #each or #map, prevents the allocated objects from being GC'd. # Maybe I'm missing something, but after this method returns nothing # should refer to the strs Array or any of the objects contained in the # Array, so GC should proceed without issue. What gives? strs.each(&:length) if should_cling strs.each {|x| char_count += x.length } char_count end # Helper to print object allocation stats. def object_stats(tag) puts "#{tag}:" puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}" puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}" end def print_with_stats(char_count) object_stats("Before GC") # Run the garbage collector. GC.start object_stats("After GC") puts char_count end end def wrapper clinger = ClingyObjects.new puts "Non-clingy:" count = clinger.generate clinger.print_with_stats(count) puts "\nClingy:" count = clinger.generate(:should_cling) clinger.print_with_stats(count) # Try to GC again for fun puts "\nTry GC again" GC.start clinger.print_with_stats(count) puts "\nDitch clinger and try GC again" clinger = nil 5.times do GC.start puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}" puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}" puts "\nSleep a bit and try again" sleep 3 end puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}" puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}" end wrapper puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}" puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}" ~~~ Output from 1.9.3-p551, 2.1.2, 2.1.3, 2.1.5, 2.1.7: ~~~ # Non-clingy: # Before GC: # TrackedArray: 1 # TrackedString: 30000 # After GC: # TrackedArray: 0 # TrackedString: 0 # 150000000 # Clingy: # Before GC: # TrackedArray: 1 # TrackedString: 30000 # After GC: # TrackedArray: 1 # TrackedString: 30000 # 150000000 # Try GC again # Before GC: # TrackedArray: 1 # TrackedString: 30000 # After GC: # TrackedArray: 1 # TrackedString: 30000 # 150000000 # Ditch clinger and try GC again # TrackedArray: 1 # TrackedString: 30000 # Sleep a bit and try again # TrackedArray: 1 # TrackedString: 30000 # Sleep a bit and try again # TrackedArray: 1 # TrackedString: 30000 # Sleep a bit and try again # TrackedArray: 1 # TrackedString: 30000 # Sleep a bit and try again # TrackedArray: 1 # TrackedString: 30000 ~~~ Output from 2.2.0 (expected output): ~~~ Non-clingy: Before GC: TrackedArray: 1 TrackedString: 30000 After GC: TrackedArray: 0 TrackedString: 0 150000000 Clingy: Before GC: TrackedArray: 1 TrackedString: 30000 After GC: TrackedArray: 0 TrackedString: 0 150000000 Try GC again Before GC: TrackedArray: 0 TrackedString: 0 After GC: TrackedArray: 0 TrackedString: 0 150000000 Ditch clinger and try GC again TrackedArray: 0 TrackedString: 0 Sleep a bit and try again TrackedArray: 0 TrackedString: 0 Sleep a bit and try again TrackedArray: 0 TrackedString: 0 Sleep a bit and try again TrackedArray: 0 TrackedString: 0 Sleep a bit and try again TrackedArray: 0 TrackedString: 0 Sleep a bit and try again TrackedArray: 0 TrackedString: 0 TrackedArray: 0 TrackedString: 0 ~~~ Thanks in advance! -- https://bugs.ruby-lang.org/