From: "bozhidar (Bozhidar Batsov)" <bozhidar@...>
Date: 2013-05-09T18:08:16+09:00
Subject: [ruby-core:54879] [ruby-trunk - Bug #8383] Ripper.lex does not handle correctly symbols whose identifiers coincide with keyword names


Issue #8383 has been updated by bozhidar (Bozhidar Batsov).


On a related note I don't think this works correctly with method definitions as well:

[30] pry(main)> Ripper.lex('def alias(arg)')
=> [[[1, 0], :on_kw, "def"],
 [[1, 3], :on_sp, " "],
 [[1, 4], :on_kw, "alias"],
 [[1, 9], :on_lparen, "("],
 [[1, 10], :on_ident, "arg"],
 [[1, 13], :on_rparen, ")"]]

[31] pry(main)> Ripper.lex('def aliass(arg)')
=> [[[1, 0], :on_kw, "def"],
 [[1, 3], :on_sp, " "],
 [[1, 4], :on_ident, "aliass"],
 [[1, 10], :on_lparen, "("],
 [[1, 11], :on_ident, "arg"],
 [[1, 14], :on_rparen, ")"]]
----------------------------------------
Bug #8383: Ripper.lex does not handle correctly symbols whose identifiers coincide with keyword names
https://bugs.ruby-lang.org/issues/8383#change-39218

Author: bozhidar (Bozhidar Batsov)
Status: Open
Priority: Normal
Assignee: 
Category: lib
Target version: 
ruby -v: ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin12.3.0]
Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN


This problem is best illustrated with an example

    [23] pry(main)> Ripper.lex(':one')
    => [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_ident, "one"]]
    [24] pry(main)> Ripper.lex(':alias')
    => [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_kw, "alias"]]

I'm pretty sure that the correct tokenization of the second example should yield *:on_ident* instead of *:on_kw*. Or am I missing something?

This is the behaviour on both 1.9.3 and 2.0.0.


-- 
http://bugs.ruby-lang.org/