From: nov@...
Date: 2020-12-16T10:02:52+00:00
Subject: [ruby-core:101466] [Ruby master Bug#17345] ripper: nothing raised when assigning to keyword variables

Issue #17345 has been updated by no6v (Nobuhiro IMAI).


Sharing error messages between ruby core and ripper is fine, thank you!
Now, the '#tok' on some error events are `Elem` instead of string.

```
$ ./ruby -rripper -ve 'pp Ripper::Lexer.new("alias $x $1").scan'
ruby 3.0.0dev (2020-12-16T07:17:54Z master c668772b14) [x86_64-linux]
[#<Ripper::Lexer::Elem: on_kw@1:0 FNAME|FITEM token: "alias">,
 #<Ripper::Lexer::Elem: on_sp@1:5 FNAME|FITEM token: " ">,
 #<Ripper::Lexer::Elem: on_gvar@1:6 END token: "$x">,
 #<Ripper::Lexer::Elem: on_sp@1:8 END token: " ">,
 #<Ripper::Lexer::Elem: on_backref@1:9 END token: "$1">,
 #<Ripper::Lexer::Elem:
   on_alias_error@1:11                                               # <-
   END
   token: #<Ripper::Lexer::Elem: on_gvar@1:6 END token: "$x">        # <-
   message: can't make alias for the number variables>]
```

A quick patch is below (also fixes `lineno` and `column` as well):

```diff
diff --git a/ext/ripper/lib/ripper/lexer.rb b/ext/ripper/lib/ripper/lexer.rb
index 6a9a9cb39a..26a53a00ce 100644
--- a/ext/ripper/lib/ripper/lexer.rb
+++ b/ext/ripper/lib/ripper/lexer.rb
@@ -190,8 +190,9 @@ def _push_token(tok)
       e
     end
 
-    def on_error(mesg, tok = token())
-      @errors.push Elem.new([lineno(), column()], __callee__, tok, state(), mesg)
+    def on_error(mesg, tok = nil)
+      lineno, column, token = tok ? [*tok.pos, tok.tok] : [lineno(), column(), token()]
+      @errors.push Elem.new([lineno, column], __callee__, token, state(), mesg)
     end
     PARSER_EVENTS.grep(/_error\z/) do |e|
       alias_method "on_#{e}", :on_error
```

In particular, the `#tok` on `on_alias_error` may be wrong.
It should be `$1` in above example.

```
$ ./ruby -e 'alias $x $1' 2>&1 | cat
-e:1: can't make alias for the number variables
alias $x $1
         ^~
```


----------------------------------------
Bug #17345: ripper: nothing raised when assigning to keyword variables
https://bugs.ruby-lang.org/issues/17345#change-89238

* Author: no6v (Nobuhiro IMAI)
* Status: Closed
* Priority: Normal
* ruby -v: ruby 3.0.0dev (2020-11-25T04:36:39Z master 00f046ef57) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
```ruby
require "ripper"

Ripper.tokenize("retry=1", raise_errors: true)
# => SyntaxError (syntax error, unexpected '=', expecting end-of-input)

Ripper.tokenize("nil=1", raise_errors: true) # => ["nil", "=", "1"]
lexer = Ripper::Lexer.new("nil=1")
lexer.tokenize # => ["nil", "=", "1"]
lexer.error? # => true
lexer.errors # => []
```

Lexer recognizes there was an error, but nothing is set to `errors`.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>