[ruby-core:87352] [Ruby trunk Feature#14808] Last token of endless range should have EXPR_END

Issue #14808 has been reported by aycabta (aycabta .).

----------------------------------------
Feature #14808: Last token of endless range should have EXPR_END
https://bugs.ruby-lang.org/issues/14808

* Author: aycabta (aycabta .)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
In 2.5.1:

```ruby
irb(main):001:0> require 'ripper'
irb(main):002:0> pp Ripper.lex("case 5\nwhen 3..\n  puts(true)\nend\n")
[[[1, 0], :on_kw, "case", EXPR_BEG],
 [[1, 4], :on_sp, " ", EXPR_BEG],
 [[1, 5], :on_int, "5", EXPR_END|EXPR_ENDARG],
 [[1, 6], :on_nl, "\n", EXPR_BEG],
 [[2, 0], :on_kw, "when", EXPR_BEG],
 [[2, 4], :on_sp, " ", EXPR_BEG],
 [[2, 5], :on_int, "3", EXPR_END|EXPR_ENDARG],
 [[2, 6], :on_op, "..", EXPR_BEG],
 [[2, 8], :on_ignored_nl, "\n", EXPR_BEG],
 [[3, 0], :on_sp, "  ", EXPR_BEG],
 [[3, 2], :on_ident, "puts", EXPR_ARG],
 [[3, 6], :on_lparen, "(", EXPR_BEG|EXPR_LABEL],
 [[3, 7], :on_kw, "true", EXPR_END],
 [[3, 11], :on_rparen, ")", EXPR_ENDFN],
 [[3, 12], :on_nl, "\n", EXPR_BEG],
 [[4, 0], :on_kw, "end", EXPR_END],
 [[4, 3], :on_nl, "\n", EXPR_BEG]]
```

This is invalid code in 2.5.1, so I understand this result.

In 63451:

```ruby
irb(main):001:0> require 'ripper'
irb(main):002:0> pp Ripper.lex("case 5\nwhen 3..\n  puts(true)\nend\n")
[[[1, 0], :on_kw, "case", EXPR_BEG],
 [[1, 4], :on_sp, " ", EXPR_BEG],
 [[1, 5], :on_int, "5", EXPR_END],
 [[1, 6], :on_nl, "\n", EXPR_BEG],
 [[2, 0], :on_kw, "when", EXPR_BEG],
 [[2, 4], :on_sp, " ", EXPR_BEG],
 [[2, 5], :on_int, "3", EXPR_END],
 [[2, 6], :on_op, "..", EXPR_BEG],
 [[2, 8], :on_ignored_nl, "\n", EXPR_BEG],
 [[3, 0], :on_sp, "  ", EXPR_BEG],
 [[3, 2], :on_ident, "puts", EXPR_ARG],
 [[3, 6], :on_lparen, "(", EXPR_BEG|EXPR_LABEL],
 [[3, 7], :on_kw, "true", EXPR_END],
 [[3, 11], :on_rparen, ")", EXPR_ENDFN],
 [[3, 12], :on_nl, "\n", EXPR_BEG],
 [[4, 0], :on_kw, "end", EXPR_END],
 [[4, 3], :on_nl, "\n", EXPR_BEG]]
```

This is correct code in this revision.

I think that lex_state of the last token of endless range, `[[2, 6], :on_op, "..", EXPR_BEG]`, it should be EXPR_END. Because it's the end of an argument. It's important for REPL, RDoc, and so on.

But lex_state is parser matter in parse.y. How about this for the parser of Ruby compiler?



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>