[ruby-core:97972] [Ruby master Bug#16797] Array#flatten not checking `respond_to? :to_ary`

Issue #16797 has been updated by Dan0042 (Daniel DeLorme).


Actually what happens here is pretty interesting. This is all done via `rb_check_array_type` which attempts to convert an object by using `to_ary`.

If your object defines its _own_ `respond_to?` method, it is used to determine if `to_ary` exists and should be called.

```ruby
a = Object.new
def a.method_missing(...); Object.new; end
def a.respond_to?(...); super; end
[a].flatten #=> [#<Object:0x000055ca978ce058>]

b = Object.new
def b.to_ary; bug!; end
def b.respond_to?(...); false; end
[b].flatten #=> [#<Object:0x000055ca9789eab0>]
```

Otherwise if the object only has the _default_ `respond_to?` method, its tries to call `method_missing`. NoMethodError is rescued and taken to mean that the object does not respond to `to_ary`, and the object is not flattened.
Without a NoMethodError it means the object effectively responds to `to_ary`, and the return type must be Array.

```ruby
a = Object.new
def a.method_missing(...); 42; end
[a].flatten #=> TypeError (can't convert Object to Array (Object#to_ary gives Integer))

def a.method_missing(...); [[42]]; end
[a].flatten #=> [42]

def a.method_missing(...); super; end #NoMethodError triggered via super
[a].flatten #=> [#<Object:0x00005603341f3b18>]

def a.method_missing(...); 42.foobar; end #NoMethodError is actually because of foobar
[a].flatten #=> [#<Object:0x0000560334024300>]
```


----------------------------------------
Bug #16797: Array#flatten not checking `respond_to? :to_ary`
https://bugs.ruby-lang.org/issues/16797#change-85202

* Author: UlyssesZhan (有丘 詹)
* Status: Rejected
* Priority: Normal
* ruby -v: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
```ruby
def (a = Object.new).method_missing(...)
  Object.new
end
[a].flatten # TypeError
```
It should check whether an object responds to `to_ary` before trying to convert it into an array.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>