From: ruby@... Date: 2019-01-16T16:35:40+00:00 Subject: [ruby-core:91129] [Ruby trunk Bug#15543] rb_str_set_len should clear code range Issue #15543 has been reported by nirvdrum (Kevin Menard). ---------------------------------------- Bug #15543: rb_str_set_len should clear code range https://bugs.ruby-lang.org/issues/15543 * Author: nirvdrum (Kevin Menard) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux] * Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- Calling `rb_str_set_len` on a `String` could alter the code range. I think this hasn't been much of an issue because of pure luck rather than anything that was deliberately designed. If called on a string that already has a `CR_UNKNOWN` code range, there's no problem because the correct values will lazily calculated. An example invocation that could be problematic, using helper methods for writing C API specs from the Ruby Spec Suite, looks like: ``` @str = "abcdefghij"[0..-1] @s.rb_str_set_len(@str, 1) @str.should == "a" @str.force_encoding(Encoding::UTF_8) @str.valid_encoding?.should == true @s.RSTRING_PTR_set(@str, 1, 'B'.ord) @s.RSTRING_PTR_set(@str, 2, 0x80) @s.rb_str_set_len(@str, 3) p @str @str.valid_encoding?.should == false # This line fails because the cached code range hasn't been updated. ``` The first call to `valid_encoding?` forces the code range to be calculated for the string and it's determined to be `CR_7BIT`. Then `rb_str_set_len` is called, simulating the splitting the bytes of a valid UTF-8 multi-byte character. Here, the code range is still cached as `CR_7BIT`, but the byte sequence is invalid for UTF-8. I think the fix is a simple matter of clearing the code range in `rb_str_set_len`, but I'd appreciate a review of my analysis. -- https://bugs.ruby-lang.org/ Unsubscribe: