From: "tompng (tomoya ishida)" Date: 2022-08-11T13:08:11+00:00 Subject: [ruby-core:109474] [Ruby master Bug#18962] Ripper.lex does not handle \0, ^D and ^Z correctly. Issue #18962 has been reported by tompng (tomoya ishida). ---------------------------------------- Bug #18962: Ripper.lex does not handle \0, ^D and ^Z correctly. https://bugs.ruby-lang.org/issues/18962 * Author: tompng (tomoya ishida) * Status: Open * Priority: Normal * ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- Ripper.lex does not end with \0, ^D and ^Z, and it contains weird tokens including "\u0000" ~~~ruby # for syntax ok code Ripper.lex("hello\0\0world").map{_3} #=> ["hello"] Ripper.lex("hello\0world").map{_3} #=> ["hello", "\u0000world"] Ripper.lex("hello\0@world").map{_3} #=> ["hello", "@world"] Ripper.lex("hello\0:world").map{_3} #=> ["hello", "\u0000:", "world"] Ripper.lex("hello\0))").map{_3} #=> ["hello", "\u0000)", ")"] # for syntax invalid code Ripper.lex("end\0*").map{_3} # => ["end", "\u0000*"] Ripper.lex("end\0\0*").map{_3} # => ["end"] Ripper.lex("end\0@end").map{_3} # => ["end", "@end"] ~~~ -- https://bugs.ruby-lang.org/ Unsubscribe: