From: "nagachika (Tomoyuki Chikanaga) via ruby-core" Date: 2024-07-15T13:02:37+00:00 Subject: [ruby-core:118606] [Ruby master Bug#20030] `Ripper.tokenize('"\\C-あ"')` separates encoding valid string to encoding invalid string. Issue #20030 has been updated by nagachika (Tomoyuki Chikanaga). Backport changed from 3.2: REQUIRED to 3.2: DONE ruby_3_2 commit:a804d5514c7c0608b9fb52426ec3ec738420ad29 merged revision(s) commit:d503e1b95a40e45d7767e0175de60092de4ba54e. ---------------------------------------- Bug #20030: `Ripper.tokenize('"\\C-���"')` separates encoding valid string to encoding invalid string. https://bugs.ruby-lang.org/issues/20030#change-109136 * Author: tompng (tomoya ishida) * Status: Closed * ruby -v: ruby 3.3.0dev (2023-11-30T16:23:25Z master d048bae96b) [x86_64-linux] * Backport: 3.2: DONE ---------------------------------------- ~~~ruby Ripper.tokenize '"\\C-���"' # or Ripper.tokenize "\"\\C-\u3042\"" # => ["\"", "\x81", "\x82", "\""] ~~~ I expect all tokens to be valid_encoding if the source string is valid_encoding. This is causing IRB crash when typing `"\C-���"`. -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/