From: merch-redmine@... Date: 2021-07-09T15:52:00+00:00 Subject: [ruby-core:104569] [Ruby master Bug#15993] 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir Issue #15993 has been updated by jeremyevans0 (Jeremy Evans). Status changed from Closed to Open inversion (Yura Babak) wrote in #note-8: > jeremyevans0 (Jeremy Evans) wrote in #note-7: > > This appears to be fixed starting in Ruby 2.7 (also works in 3.0): > > **Still, there is a problem.** > `require 'bundler/setup'` fails if `LOAD_PATH` or `Gem.dir` contain Cyrillic chars, the error is similar to: > ``` > incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError) > ``` > > From the trace I have prepared the **minimum reproducible case** : > 1. Put Ruby in a location where the path will contain Cyrillic chars, like `"D:\users\������\Ruby"` > 1. Prepare 2 files (saved in UTF-8 encoding) somewhere in a location where the path will contain Cyrillic chars (can be near that Ruby): > https://gist.github.com/Inversion-des/75949795cc5be707c19d31901e79d1cf > 1. Open cmd and ensure to do `chcp 1251` in the current console session. > 1. run `"[this Ruby path]" f1.rb` I was able to reproduce the issue, but only when I installed Ruby into a path not supported by the `Windows-1251` encoding: ``` d:\��������������>d:\zz-k��nnen2\Ruby31-x64\bin\bundle install --local d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `expand_path': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError) from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `expand_path' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:94:in `bundle_path' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:682:in `configure_gem_home' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:663:in `configure_gem_home_and_path' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:80:in `configure' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler.rb:193:in `definition' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli/install.rb:57:in `run' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:259:in `block in install' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/settings.rb:133:in `temporary' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:258:in `install' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:30:in `dispatch' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/cli.rb:24:in `start' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:49:in `block in ' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/3.1.0/bundler/friendly_errors.rb:130:in `with_friendly_errors' from d:/zz-k��nnen2/Ruby31-x64/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:37:in `' from d:/zz-k?nnen2/Ruby31-x64/bin/bundle:31:in `load' from d:/zz-k?nnen2/Ruby31-x64/bin/bundle:31:in `
' ``` Part of the underlying issue seems to be that `__FILE__` and `__dir__` are not UTF-8 encoded for the main script, unlike required files. I'm not sure if changing that alone will fix the issue, though. When I run the following script (`f3.rb`): ``` p ['__FILE__', __FILE__, __FILE__.encoding] p ['__dir__', __dir__, __dir__.encoding] p ['Gem.dir', Gem.dir, Gem.dir.encoding] puts 'Gem.path' Gem.path.each do |s| p [s, s.encoding] end puts '$:' $:.each do |s| p [s, s.encoding] end ``` I get the following when using Ruby installed in a non-ASCII path: ``` d:\��������������>d:\zz-k��nnen2\Ruby31-x64\bin\ruby D:\��������������\f3.rb ["__FILE__", "D:/\xC5\xE2\xE3\xE5\xED\xE8\xE9/f3.rb", #] ["__dir__", "D:/\xC5\xE2\xE3\xE5\xED\xE8\xE9", #] ["Gem.dir", "d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #] Gem.path ["C:/Users/jeremye/.gem/ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #] $: ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0/x64-ucrt", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0/x64-ucrt", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0/x64-mingw-ucrt", #] ``` and when installed into an ASCII path: ``` d:\��������������>C:\Ruby30-x64\bin\ruby d:\��������������\f3.rb ["__FILE__", "d:/\xC5\xE2\xE3\xE5\xED\xE8\xE9/f3.rb", #] ["__dir__", "d:/\xC5\xE2\xE3\xE5\xED\xE8\xE9", #] ["Gem.dir", "C:/Ruby30-x64/lib/ruby/gems/3.0.0", #] Gem.path ["C:/Users/jeremye/.gem/ruby/3.0.0", #] ["C:/Ruby30-x64/lib/ruby/gems/3.0.0", #] $: ["C:/Ruby30-x64/lib/ruby/site_ruby/3.0.0", #] ["C:/Ruby30-x64/lib/ruby/site_ruby/3.0.0/x64-msvcrt", #] ["C:/Ruby30-x64/lib/ruby/site_ruby", #] ["C:/Ruby30-x64/lib/ruby/vendor_ruby/3.0.0", #] ["C:/Ruby30-x64/lib/ruby/vendor_ruby/3.0.0/x64-msvcrt", #] ["C:/Ruby30-x64/lib/ruby/vendor_ruby", #] ["C:/Ruby30-x64/lib/ruby/3.0.0", #] ["C:/Ruby30-x64/lib/ruby/3.0.0/x64-mingw32", #] ``` It looks like the difference in the non-ASCII path case is that `ASCII-8BIT` encoding is used even if the path itself is valid UTF-8. This is true even if you force a UTF-8 code page (though that does fix `__FILE__` and `__dir__`): ``` d:\��������������>chcp 65001 Active code page: 65001 d:\��������������>d:\zz-k��nnen2\Ruby31-x64\bin\ruby D:\��������������\f3.rb ["__FILE__", "D:/��������������/f3.rb", #] ["__dir__", "D:/��������������", #] ["Gem.dir", "d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #] Gem.path ["C:/Users/jeremye/.gem/ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/gems/3.1.0", #] $: ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby/3.1.0/x64-ucrt", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/site_ruby", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby/3.1.0/x64-ucrt", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/vendor_ruby", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0", #] ["d:/zz-k\xC3\xB6nnen2/Ruby31-x64/lib/ruby/3.1.0/x64-mingw-ucrt", #] ``` Since there does appear to be an issue, I'll reopen this. Hopefully someone with more knowledge in this area can suggest a possible fix. ---------------------------------------- Bug #15993: 'require' doesn't work if there are Cyrillic chars in the path to Ruby dir https://bugs.ruby-lang.org/issues/15993#change-92849 * Author: inversion (Yura Babak) * Status: Open * Priority: Normal * ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- I���m trying to build a cross-platform portable application with Ruby onboard and there is a problem on Windows. A user usually installs it to the Roaming folder which sits inside a user folder which can often have not a Latin name or contain spaces). When there is a Cyrillic character (maybe just not Latin) in the path ��� require of any gem doesn���t work: ``` D:\users\������\Ruby\2.6\bin>ruby -v ruby 2.6.3p62 (2019-04-16 revision 67580) [x64-mingw32] D:\users\������\Ruby\2.6\bin>ruby -e "require 'logger'" Traceback (most recent call last): 1: from :2:in `' :2:in `require': No such file or directory -- D:/users/�������������/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError) D:\users\������\Ruby\2.6\bin>ruby --disable=rubyopt -e "require 'logger'" Traceback (most recent call last): 1: from :2:in `' :2:in `require': No such file or directory -- D:/users/�������������/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError) D:\users\������\Ruby\2.6\bin>gem list Traceback (most recent call last): 1: from :2:in `' :2:in `require': No such file or directory -- D:/users/�������������/Ruby/2.6/lib/ruby/2.6.0/rubygems.rb (LoadError) ``` We can see such encoding transformations in the output: ``` ������ (utf-8) == ������������� (win1251) ``` I have an old Ruby installation that works fine: ``` D:\users\������\Ruby\2.0\bin>ruby -e "require 'logger'" D:\users\������\Ruby\2.0\bin>ruby -v ruby 2.0.0p451 (2014-02-24) [i386-mingw32] ``` The same is for `ruby 2.0.0p643 (2015-02-25) [i386-mingw32]` . I also checked that require fails in the same case for `ruby 2.1.9p490 (2016-03-30 revision 54437) [i386-mingw32]` -- https://bugs.ruby-lang.org/ Unsubscribe: