[ruby-core:25875] [Bug #1939] Ripper doesn't handle local variables

Issue #1939 has been updated by Magnus Holm.


Since nobody seems to be working on this, I thought I might give it a try. I should warn you that I really don't know parse.y, Bison or C, but it sure was fun trying to figure it all out! Please correct me if I've got something wrong.

The first issue is that Ripper doesn't have a local variable-table at all. This isn't difficult at all to fix; you only need to remove a few #ifndef RIPPER, and move some function calls out of /*%%%*/. It looks like these functions must be called in both Ruby and Ripper: shadowing_lvar, local_push, local_pop, arg_var, local_var, dyna_push, dyna_pop and new_bv. We must also add something like assignable() which only adds the var in the lvtbl without returning a NODE. Seems correct? If so, I've already done this.

The other issue is a little more complicated: In Ripper yylvar.val is set to the result of the scanner event and we discard the ID. This is because we want some of the parser events to include scanner events, but this also means we for instance can't call local_var($1), because it's a VALUE, not an ID. This is also a VALUE we can't control, because the user will probably redefine it in his SexpBuilder. 

So it looks like we want to have two values in Ripper: The VALUE returned from the callback and the ID returned from the lexer (only for scanner event. nil otherwise). As far as I can see, it means we'll have to define YYSTYPE ourself, remove yylvar-declarations from %token and handle it all ourself. This means we'll have to do this:
* Rewrite ripper_dispatch0-5 to call .val on all the values, and alloc a YYSTYPE where .val is set to the result of the funcall.
* Add a macro like this in Ripper: #define GET_ID(x) x.id
* And something like this in Ruby: #define GET_ID(x) x
* Define all the macros I listed above (shadowing_lvar etc.) to call GET_ID(id):
  #define local_var(id) local_var_gen(parser, GET_ID(id));
* Make sure the lexer sets yylvar.id and ripper_dispatch_scan_event sets yylvar.val in Ripper.

Then, in my world, this would work :-)

However, I assume there is a much easier way. Any Bison/parse.y-gurus know a better solution?

If not, I would love to try to write a patch.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1939

----------------------------------------
http://redmine.ruby-lang.org