[ruby-core:113938] [Ruby master Bug#19736] Ripper.tokenize ignores token after heredoc_beg when embexpr inside heredoc is unclosed
From:
"tompng (tomoya ishida) via ruby-core" <ruby-core@...>
Date:
2023-06-19 10:27:36 UTC
List:
ruby-core #113938
Issue #19736 has been reported by tompng (tomoya ishida).
----------------------------------------
Bug #19736: Ripper.tokenize ignores token after heredoc_beg when embexpr inside heredoc is unclosed
https://bugs.ruby-lang.org/issues/19736
* Author: tompng (tomoya ishida)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.3.0dev (2023-06-19T09:28:10Z master 2a80bac9f0) [x86_64-linux]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
I expect Ripper.tokenize(code) to tokenize `+1+2+3\n` part.
```ruby
code = <<'RUBY'
<<A+1+2+3
#{4
RUBY
Ripper.tokenize code
# => ["<<A", "\#{", "4", "\n"]
```
It will be easy for IRB's internal code to handle indent and other things if Ripper.tokenize can tokenize all part of the code even if it is incomplete.
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/