From: lamont@... Date: 2020-03-09T19:49:45+00:00 Subject: [ruby-core:97427] [Ruby master Bug#16683] Regression in ruby 2.7 opening filenames that have non-UTF8 characters in them Issue #16683 has been reported by lamont (Lamont Granquist). ---------------------------------------- Bug #16683: Regression in ruby 2.7 opening filenames that have non-UTF8 characters in them https://bugs.ruby-lang.org/issues/16683 * Author: lamont (Lamont Granquist) * Status: Open * Priority: Normal * ruby -v: ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-darwin18] * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- Real simple repro case: ``` [1] pry(main)> fp = File.open("/tmp/chef-test-\xFDmlaut", "w+") Errno::EILSEQ: Illegal byte sequence @ rb_sysopen - /tmp/chef-test-���mlaut from (pry):1:in `initialize' ``` This used to work in ruby < 2.6, setting Encoding.default_external and Encoding.default_internal to ASCII-8BIT does not help. Seems to fail on every linux based O/S we test on along with MacOS at least. This is a use case that we need to support in cases where ruby needs to run under UTF-8 but may need to manage files with names that were written in other encodings. The ruby application does not necessarily control or own these files or their names and cannot impose UTF-8 naming on the filesystem. The requirement is that the ruby application is able to read any arbitrary file on the filesystem written by any other application on the system, with any encoding set locally on that other application (where the ruby process has no a prior information about which application wrote the file, it just has the file). And even if the filename is just binary garbage the ruby application needs to be able to open it and analyze it. Since this is getting thrown deep from rb_sysopen I'm not sure of any way to work around it from within ruby by finding an alternative API. -- https://bugs.ruby-lang.org/ Unsubscribe: