[ruby-core:109474] [Ruby master Bug#18962] Ripper.lex does not handle \0, ^D and ^Z correctly.
From:
"tompng (tomoya ishida)" <noreply@...>
Date:
2022-08-11 13:08:11 UTC
List:
ruby-core #109474
Issue #18962 has been reported by tompng (tomoya ishida).
----------------------------------------
Bug #18962: Ripper.lex does not handle \0, ^D and ^Z correctly.
https://bugs.ruby-lang.org/issues/18962
* Author: tompng (tomoya ishida)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Ripper.lex does not end with \0, ^D and ^Z, and it contains weird tokens including "\u0000"
~~~ruby
# for syntax ok code
Ripper.lex("hello\0\0world").map{_3} #=> ["hello"]
Ripper.lex("hello\0world").map{_3} #=> ["hello", "\u0000world"]
Ripper.lex("hello\0@world").map{_3} #=> ["hello", "@world"]
Ripper.lex("hello\0:world").map{_3} #=> ["hello", "\u0000:", "world"]
Ripper.lex("hello\0))").map{_3} #=> ["hello", "\u0000)", ")"]
# for syntax invalid code
Ripper.lex("end\0*").map{_3} # => ["end", "\u0000*"]
Ripper.lex("end\0\0*").map{_3} # => ["end"]
Ripper.lex("end\0@end").map{_3} # => ["end", "@end"]
~~~
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>