[ruby-core:100891] [Ruby master Feature#17276] Ripper stops tokenizing after keyword as a method parameter

Issue #17276 has been updated by Eregon (Benoit Daloze).


jeremyevans0 (Jeremy Evans) wrote in #note-1:
> Maybe we could support keyword arguments in `Ripper.lex` and `Ripper.tokenize` to raise SyntaxError for errors?  Here's a pull request for that approach: https://github.com/ruby/ruby/pull/3774

I agree it would be nice.

Do you think the same would be possible for `Ripper.sexp/sexp_raw`?
Currently it just returns `nil` if there is some error, which is unhelpful if one wants to know why it failed to lex/parse:
```
> Ripper.sexp('def n')
=> nil
```

----------------------------------------
Feature #17276: Ripper stops tokenizing after keyword as a method parameter
https://bugs.ruby-lang.org/issues/17276#change-88539

* Author: no6v (Nobuhiro IMAI)
* Status: Open
* Priority: Normal
----------------------------------------
Although these are obviously syntax errors at this time, the following
codes cannot be tokenized correctly by `Ripper.tokenize`.

```
$ cat src.rb
def req(true) end
def opt(true=0) end
def rest(*true) end
def keyrest(**true) end
def block(&true) end
->true{}
->true=0{}
->*true{}
->**true{}
->&true{}
$ ruby -rripper -vlne 'p Ripper.tokenize($_)' src.rb
ruby 3.0.0dev (2020-10-21T00:24:47Z master da25affdac) [x86_64-linux]
["def", " ", "req", "(", "true", ")"]
["def", " ", "opt", "(", "true", "=", "0", ")"]
["def", " ", "rest", "(", "*", "true", ")"]
["def", " ", "keyrest", "(", "**", "true", ")"]
["def", " ", "block", "(", "&", "true", ")"]
["->", "true", "{"]
["->", "true", "=", "0", "{"]
["->", "*", "true", "{"]
["->", "**", "true", "{"]
["->", "&", "true", "{"]
```

`end` and `}` are not shown in result.

This seems to prevent `irb` from determining the continuity of the input.
See: https://github.com/ruby/irb/issues/38




-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>