[ruby-core:75732] [Ruby trunk Bug#12431] Strange behavior of String#encode('UTF-8', 'UTF-8', ...) when the encoding of the source string is not UTF-8

Issue #12431 has been reported by Paul Grayson.

----------------------------------------
Bug #12431: Strange behavior of String#encode('UTF-8', 'UTF-8', ...) when the encoding of the source string is not UTF-8
https://bugs.ruby-lang.org/issues/12431

* Author: Paul Grayson
* Status: Open
* Priority: Normal
* Assignee: 
* ruby -v: ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
* Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
----------------------------------------
When the dst_encoding and src_encoding options of String#encode are the same, it appears to ignore the encoding given and instead operate on the actual encoding of the string.  Examples:

~~~
"abcdÁ".force_encoding('ASCII').encode('UTF-8', 'UTF-8', invalid: :replace, undef: :replace)
=> "abcd??"

"abcdÁ".force_encoding('ASCII').encode('UTF-8', 'UTF-8', invalid: :replace, undef: :replace, replace: '�')
Encoding::CompatibilityError: incompatible character encodings: US-ASCII and UTF-8

"abcdÁ\xff".encode('ASCII', 'ASCII', invalid: :replace, undef: :replace).force_encoding('UTF-8')
=> "abcdÁ�"
~~~

Also, without the "replace" options, exceptions are not raised as they should be:

~~~
"\xff".force_encoding('ASCII').encode('UTF-8', 'UTF-8')
=> "\xFF"
~~~

I looked a little at the code, and I think the problem might be in [this block](https://github.com/ruby/ruby/blob/v2_3_1/transcode.c#L2697-L2709) where the given string is passed to `rb_str_scrub` without any other encoding information.

What I would expect is for `s.dup.force_encoding('X').encode('Y', opts)` to behave identically to `s.encode('Y', 'X', opts)`, but that is clearly not the case.

Verified on Ruby 2.1.5, 2.3.0, and 2.3.1.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>