From: "tompng (tomoya ishida)" <noreply@...>
Date: 2022-08-12T22:26:12+00:00
Subject: [ruby-core:109476] [Ruby master Bug#18963] Ripper.tokenize(code).join != code when heredoc and multiline string is on the same line

Issue #18963 has been reported by tompng (tomoya ishida).

----------------------------------------
Bug #18963: Ripper.tokenize(code).join != code when heredoc and multiline string is on the same line
https://bugs.ruby-lang.org/issues/18963

* Author: tompng (tomoya ishida)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-darwin20]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Expect Ripper.tokenize(code).join to match the original code but it doesn't.

~~~ruby
code = %[
<<A + "hello
A
world"
]
eval code # => "ruby\nhello\nworld"
code == Ripper.tokenize(code).join # => false
eval Ripper.tokenize(code).join # => Syntax Error
~~~

Other codes that does not match.
`"<<A;/a\nA\nb/"` `"<<A;:'a\nA\nb'"` `"<<A;%[a\nA\nb]"`

IRB fails coloring those codes.




-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>