[ruby-core:102608] [Ruby master Bug#16970] Encoding of ENV value returns ASCII-8BIT in Ruby2.6 or later

Issue #16970 has been updated by jeremyevans0 (Jeremy Evans).

Status changed from Open to Closed

Ruby 3.0 uses UTF-8 for ENV values on Windows by default, even if the code page is not UTF-8.  So I think this and #16623 can be closed.

----------------------------------------
Bug #16970: Encoding of ENV value returns ASCII-8BIT in Ruby2.6 or later
https://bugs.ruby-lang.org/issues/16970#change-90594

* Author: masuyama (Soichi Masuyama)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 2.6.5p114 [i386-mingw32], ruby 2.7.1p83 [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
## Problem Report

When internal encoding is set to UTF-8, encoding of ENV value always return ASCII-8BIT. (UTF-8 is expected).

## Result of reproduce process

Ruby 2.5
``` ruby
set TEST=日本
ruby --encoding=UTF-8:UTF-8 -e "p 'test'.encoding"       #=> #<Encoding:UTF-8>
ruby --encoding=UTF-8:UTF-8 -e "p ENV['TEST'].encoding"  #=> #<Encoding:UTF-8>

```
Ruby 2.6
``` ruby
set TEST=日本
ruby --encoding=UTF-8:UTF-8 -e "p 'test'.encoding"       #=> #<Encoding:UTF-8>
ruby --encoding=UTF-8:UTF-8 -e "p ENV['TEST'].encoding"  #=> #<Encoding:ASCII-8BIT>    **INVALID RESULT** Expected Result is UTF-8.
```

``` ruby
ENV['TEST'] =~ /日本(nonascii)/  # => incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)    
```


## Regression occurred in the following Ticket

fallback env encoding to ASCII-8BIT
https://github.com/ruby/ruby/commit/7f0d337be73bb2465b40009fe23f3b7be6b0dc90


## Reason of bug

rb_str_cat_conv_enc_opts return Qnil when from/to encoding is the same (UTF-8) and fallback code
introduced by the following commit set the encoding to ASCII-8BIT.

https://github.com/ruby/ruby/commit/7f0d337be73bb2465b40009fe23f3b7be6b0dc90


## Fixes

When internel encoding is UTF-8, return the string as is since there is no need to call encoding conversion.
https://github.com/ruby/ruby/pull/3239




-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>