[ruby-core:114381] [Ruby master Bug#19838] Ripper nested heredocs
From:
"kddnewton (Kevin Newton) via ruby-core" <ruby-core@...>
Date:
2023-08-12 01:25:37 UTC
List:
ruby-core #114381
Issue #19838 has been reported by kddnewton (Kevin Newton).
----------------------------------------
Bug #19838: Ripper nested heredocs
https://bugs.ruby-lang.org/issues/19838
* Author: kddnewton (Kevin Newton)
* Status: Open
* Priority: Normal
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
I'm seeing some odd behavior on nested heredocs in the ripper lexer.
``` ruby
<<~H1
1
#{<<~H2}
2
H2
3
H1
```
$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'
``` ruby
[[[1, 0], :on_heredoc_beg, "<<~H1", BEG],
[[1, 5], :on_nl, "\n", BEG],
[[2, 0], :on_ignored_sp, " ", BEG],
[[2, 2], :on_tstring_content, "1\n", BEG],
[[3, 0], :on_ignored_sp, " ", BEG],
[[3, 2], :on_embexpr_beg, "\#{", BEG],
[[3, 4], :on_heredoc_beg, "<<~H2", BEG],
[[3, 9], :on_embexpr_end, "}", END],
[[3, 10], :on_tstring_content, "\n", BEG],
[[4, 0], :on_ignored_sp, " ", BEG],
[[4, 4], :on_tstring_content, "2\n", BEG],
[[5, 0], :on_heredoc_end, " H2\n", BEG],
[[6, 0], :on_ignored_sp, " ", BEG],
[[6, 2], :on_tstring_content, "3\n", BEG],
[[7, 0], :on_heredoc_end, "H1\n", BEG]]
```
This seems totally correct. But if you change it to a non-indenting heredoc:
``` ruby
<<-H1
1
#{<<~H2}
2
H2
3
H1
```
$ cat test.rb | ruby -rripper -e 'pp Ripper.lex(ARGF.read)'
``` ruby
[[[1, 0], :on_heredoc_beg, "<<-H1", BEG],
[[1, 5], :on_nl, "\n", BEG],
[[2, 0], :on_tstring_content, " 1\n" + " ", BEG],
[[3, 2], :on_embexpr_beg, "\#{", BEG],
[[3, 4], :on_heredoc_beg, "<<~H2", BEG],
[[3, 9], :on_embexpr_end, "}", END],
[[3, 10], :on_tstring_content, "\n" + " 3\n", BEG],
[[4, 0], :on_ignored_sp, " ", BEG],
[[4, 4], :on_tstring_content, "2\n", BEG],
[[5, 0], :on_heredoc_end, " H2\n", BEG],
[[7, 0], :on_heredoc_end, "H1\n", BEG]]
```
Now the newline on line 3 gets concatenated with the content from line 6, and that token is dropped from the list entirely. This seems unintentional, but I'm not sure.
--
https://bugs.ruby-lang.org/
______________________________________________
ruby-core mailing list -- ruby-core@ml.ruby-lang.org
To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/