[ruby-core:104567] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir

Issue #15993 has been updated by inversion (Yura Babak).


jeremyevans0 (Jeremy Evans) wrote in #note-7:
> This appears to be fixed starting in Ruby 2.7 (also works in 3.0):
 
**Still, there is a problem.**
`require 'bundler/setup'` fails if `LOAD_PATH` or `Gem.dir` contain Cyrillic chars, the error is similar to:
```
incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
```

From the trace I have prepared the **minimum reproducible case** :
1. Put Ruby in a location where the path will contain Cyrillic chars, like `"D:\users\киї\Ruby"`
1. Prepare 2 files (saved in UTF-8 encoding) somewhere in a location where the path will contain Cyrillic chars (can be near that Ruby):
   https://gist.github.com/Inversion-des/75949795cc5be707c19d31901e79d1cf
1. Open cmd and ensure to do `chcp 1251` in the current console session.
1. run `"[this Ruby path]" f1.rb`

You will see that the same `__dir__` output is different between files (f2 is required). If you will try to run f2.rb — output will be the same as for f1. So, **require_relative somehow changes the encoding** here.

To emulate problems with the 'bundler/setup' there are next lines:
``` ruby
# fails: incompatible character encodings: Windows-1251 and UTF-8 (Encoding::CompatibilityError)
p start_with:$LOAD_PATH[0].start_with?(__dir__)
# fails: incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
p start_with:$LOAD_PATH[0].start_with?(Gem.dir)
```

To see the real problem you should comment these lines and also prepare next files (I'm not sure content is important by add at least one gem there)
* Gemfile
* Gemfile.lock

And to see both problems there should also be the `.bundle\config` file with a line like:
`BUNDLE_PATH: "../platform/Ruby_gems"`

In the `bundler\settings.rb` it will use `explicit_path` if the `BUNDLE_PATH` defined and `Bundler.rubygems.gem_dir` otherwise.


Workaround to overcome both errors you can find in the f1.rb in the related commented section:
``` ruby
Gem.dir.force_encoding 'UTF-8'
Gem.path.each {|path| path.force_encoding 'UTF-8' }
if $:[0].encoding.name == 'Windows-1251'
    $:.each {|path| path.encode! 'UTF-8' }
    $:.push '.'    # somehow it helps, looks like a modification of array is needed
end

```

My environment:
* Windows10 Pro
* Ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x64-mingw32]
* Bundler version 2.2.22
* RubyGems version 3.2.22

----------------------------------------
Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir
https://bugs.ruby-lang.org/issues/15993#change-92846

* Author: inversion (Yura Babak)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
I’m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. 
A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). 
When there is a Cyrillic character (maybe just not Latin) in the path — require of any gem doesn’t work:

```
D:\users\киї\Ruby\2.6\bin>ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32]

D:\users\киї\Ruby\2.6\bin>ruby -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'"
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

D:\users\киї\Ruby\2.6\bin>gem list
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': No such file or directory -- D:/users/РєРёС—/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError)

```

We can see such encoding transformations in the output:

```
киї (utf-8) == РєРёС— (win1251)
```

I have an old Ruby installation that works fine:

```
D:\users\киї\Ruby\2.0\bin>ruby -e "require 'logger'"

D:\users\киї\Ruby\2.0\bin>ruby -v
ruby 2.0.0p451 (2014-02-24) [i386-mingw32]
```
The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` .

I also checked that require fails in the same case for
`ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]`



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>