[ruby-core:23626] [Bug #1535] Hash#merge! Inside Iterator Can Cause RuntimeError

Issue #1535 has been updated by Run Paint Run Run.


> I believe that calling Hash#merge inside a Hash#each block is already prohibited in 1.8.

I'm afraid I do not understand. It is not prohibited in the sense it is made impossible, because the following code works as one would expect on 1.8.7: 

    >> x={:glark => :quark}
    => {:glark=>:quark}
    >> h = {:foo=>:bar}
    => {:foo=>:bar}
    >> h.each { h = h.merge(x) }
    => {:foo=>:bar}
    >> p h
    {:foo=>:bar, :glark=>:quark}
    => nil

Or:

    >> h = {:foo=>:bar}
    => {:foo=>:bar}
    >> x={:glark => :quark}
    => {:glark=>:quark}
    >> h.each { h.merge!(x) }
    => {:foo=>:bar, :glark=>:quark}

It is not prohibited in the sense that it is described as inadvisable in the documentation, either.

> A detailed info on your "very specific conditions" can help us a lot.

The information given in the original bug report was really all I could ascertain. I tried running under strace and a debugger, but couldn't crystallize the problem further. Maybe some worked examples under 1.8.7 IRB will illuminate the problem.

Using the example given in my original report:

>> hash = {1 => 2, 3 => 4, 5 => 6}
=> {5=>6, 1=>2, 3=>4}
>> big_hash = {}
=> {}
>> 64.times { |k| big_hash[k.to_s] = k }
=> 64
>> hash.each { hash.merge!(big_hash) }
RuntimeError: hash modified during iteration
	from (irb):4:in `each'
	from (irb):4

So, with those specific parameters, calling #merge! from with #each raises a RuntimeError. If this usage really was prohibited, that's what you would expect. However, if we modify the example so _hash_ only has 4 elements the code runs as follows:

>> hash = {1 => 2, 3 => 4}
=> {1=>2, 3=>4}
>> big_hash = {}
=> {}
>> 64.times { |k| big_hash[k.to_s] = k }
=> 64
>> hash.each { hash.merge!(big_hash) }
=> {"6"=>6, "11"=>11, "22"=>22, "33"=>33, "44"=>44, "55"=>55, "7"=>7, "12"=>12, "23"=>23, "34"=>34, "45"=>45, "56"=>56, "8"=>8, "13"=>13, "24"=>24, "35"=>35, "46"=>46, "57"=>57, "9"=>9, "14"=>14, "25"=>25, "36"=>36, "47"=>47, "58"=>58, 1=>2, "15"=>15, "26"=>26, "37"=>37, "48"=>48, "59"=>59, "60"=>60, "0"=>0, "16"=>16, "27"=>27, "38"=>38, "49"=>49, "50"=>50, "61"=>61, "1"=>1, "17"=>17, "28"=>28, "39"=>39, "40"=>40, "51"=>51, "62"=>62, "2"=>2, "18"=>18, "29"=>29, "30"=>30, "41"=>41, "52"=>52, "63"=>63, 3=>4, "3"=>3, "19"=>19, "20"=>20, "31"=>31, "42"=>42, "53"=>53, "4"=>4, "10"=>10, "21"=>21, "32"=>32, "43"=>43, "54"=>54, "5"=>5}

Here, no RuntimeError was raised despite the only difference between the two examples being the size of _hash_. 

Similarly, observe the result if we modify the original example by populating _big_hash_ with only 63 entries, keeping everything else constant:

>> hash = {1 => 2, 3 => 4, 5 => 6}
=> {5=>6, 1=>2, 3=>4}
>> big_hash = {}
=> {}
>> 63.times { |k| big_hash[k.to_s] = k }
=> 63
>> hash.each { hash.merge!(big_hash) }
=> {"6"=>6, "11"=>11, "22"=>22, "33"=>33, "44"=>44, "55"=>55, 5=>6, "7"=>7, "12"=>12, "23"=>23, "34"=>34, "45"=>45, "56"=>56, "8"=>8, "13"=>13, "24"=>24, "35"=>35, "46"=>46, "57"=>57, "9"=>9, "14"=>14, "25"=>25, "36"=>36, "47"=>47, "58"=>58, 1=>2, "15"=>15, "26"=>26, "37"=>37, "48"=>48, "59"=>59, "60"=>60, "0"=>0, "16"=>16, "27"=>27, "38"=>38, "49"=>49, "50"=>50, "61"=>61, "1"=>1, "17"=>17, "28"=>28, "39"=>39, "40"=>40, "51"=>51, "62"=>62, "2"=>2, "18"=>18, "29"=>29, "30"=>30, "41"=>41, "52"=>52, 3=>4, "3"=>3, "19"=>19, "20"=>20, "31"=>31, "42"=>42, "53"=>53, "4"=>4, "10"=>10, "21"=>21, "32"=>32, "43"=>43, "54"=>54, "5"=>5}

Again, the code behaves as we expect. However, if we adjust the above example so that big_hash has 65 entries, the RuntimError is raised again: 

>> hash = {1 => 2, 3 => 4, 5=>6}
=> {5=>6, 1=>2, 3=>4}
>> big_hash = {}
=> {}
>> 65.times { |k| big_hash[k.to_s] = k }
=> 65
>> hash.each { hash.merge!(big_hash) }
RuntimeError: hash modified during iteration
	from (irb):38:in `each'
	from (irb):38
	from :0

This is what I meant by "very specific conditions". I have no further insight or explanation into their cause. :-(

The bug, then, is that the RuntimeError is unexpected. Code that worked correctly one day, would start crashing another just because the dimensions of the data set changed. If Hash#merge! is generally dangerous inside iterator blocks, its usage should be prohibited consistently such that all of the above examples raise a RuntimeError, and the documentation must make this limitation clear. This is what happens when Hash#rehash is called inside of an iterator block: any and all usages of this combination raise a RuntimeError.

Thank you for looking into this, Shyouhei, I'm sorry I can't be of any more assistance.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1535

----------------------------------------
http://redmine.ruby-lang.org