[ruby-core:71013] [Ruby trunk - Bug #11572] [Open] Urnary operator causing references to unreachable objects in 2.1.x ?

Issue #11572 has been reported by Danny Guinther.

----------------------------------------
Bug #11572: Urnary operator causing references to unreachable objects in 2.1.x ?
https://bugs.ruby-lang.org/issues/11572

* Author: Danny Guinther
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
Perhaps this is an error on my part, but I stumbled across some weird GC behavior related to the unary & (ampersand) operator on 2.1.x.

I don't have any leads as to what the cause of the issue might be, but the gist of the issue is that using & with Array#each or Array#map seems to cause references to unreachable objects to be maintained, preventing those unreferenced objects from being GC'd.

The majority of my testing has been on Ubuntu 14.04.3, though a colleague was kind enough to verify that the behavior also occurs on OSX.

This seems like it is likely related to https://github.com/ruby/ruby/pull/592 which was ultimately solved by commit 2f3b28c682fe3010ed3b8803199616c12b52512d:

+Sat Apr 12 22:11:10 2014  Nobuyoshi Nakada  <[e-mail]>
+
+       * string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear
+         caller's self which is useless, so that it can get collected.
+         [Fixes GH-592]

As far as I can tell, this commit was not backported to 2.1.x. If this commit did fix the issue, should it be backported to 2.1? I haven't seen been able to find an existing bug for this issue if one exists, so it's unclear to me why this wouldn't have been backported.

I've been using the script below to experiment with the phenomenon.
I also made a gist of the script here: https://gist.github.com/tdg5/0b9f145edb5114a2dca1

~~~
# Create some special classes to facilitate tracking allocated objects.
class TrackedArray < Array; end
class TrackedString < String; end
STRANG = "a" * 5000

class ClingyObjects
  def generate(should_cling = false)
    strs = TrackedArray.new
    30000.times { strs << TrackedString.new(STRANG) }

    char_count = 0
    # I'm not sure why, but using the unary & operator on the Array, whether
    # through #each or #map, prevents the allocated objects from being GC'd.
    # Maybe I'm missing something, but after this method returns nothing
    # should refer to the strs Array or any of the objects contained in the
    # Array, so GC should proceed without issue. What gives?

    strs.each(&:length) if should_cling
    strs.each {|x| char_count += x.length }
    char_count
  end

  # Helper to print object allocation stats.
  def object_stats(tag)
    puts "#{tag}:"
    puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
    puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
  end

  def print_with_stats(char_count)
    object_stats("Before GC")
    # Run the garbage collector.
    GC.start
    object_stats("After GC")
    puts char_count
  end
end

def wrapper
  clinger = ClingyObjects.new
  puts "Non-clingy:"
  count = clinger.generate
  clinger.print_with_stats(count)
  puts "\nClingy:"
  count = clinger.generate(:should_cling)
  clinger.print_with_stats(count)
  # Try to GC again for fun
  puts "\nTry GC again"
  GC.start
  clinger.print_with_stats(count)
  puts "\nDitch clinger and try GC again"
  clinger = nil
  5.times do
    GC.start
    puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
    puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
    puts "\nSleep a bit and try again"
    sleep 3
  end
  puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
  puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
end
wrapper
puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
~~~

Output from 1.9.3-p551, 2.1.2, 2.1.3, 2.1.5, 2.1.7:

~~~
# Non-clingy:
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 0
# TrackedString: 0
# 150000000

# Clingy:
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 1
# TrackedString: 30000
# 150000000

# Try GC again
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 1
# TrackedString: 30000
# 150000000

# Ditch clinger and try GC again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000
~~~

Output from 2.2.0 (expected output):

~~~
Non-clingy:
Before GC:
TrackedArray: 1
TrackedString: 30000
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Clingy:
Before GC:
TrackedArray: 1
TrackedString: 30000
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Try GC again
Before GC:
TrackedArray: 0
TrackedString: 0
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Ditch clinger and try GC again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0
TrackedArray: 0
TrackedString: 0
~~~

Thanks in advance!



-- 
https://bugs.ruby-lang.org/