From: Ryan Davis <ryand-ruby@...>
Date: 2012-05-16T08:17:06+09:00
Subject: Re: Command vs method in Ripper


On May 13, 2012, at 16:53 , Jeremy W. wrote:

> Hi guys,
> 
> I've been playing around with Ripper for the first time today.  Parsing
> method calls with and without parentheses gives me different output and
> I'm not sure why. When using brackets, Ripper outputs a 'method',
> without them gives me a 'command'.
> 
> Are these actually different things inside Ruby, or is this just a
> difference in the way Ripper outputs them?
> 
> If there is a internal difference between them, could someone please
> point me to either some more information, or the relevant bit of the
> Ruby source code I need to read to learn more.
> 
> If they are not different internally, what is the reason that Ripper
> displays them as such?

Because ripper shows too much detail about the way the parse tree is engineered and not about the semantics of the language itself. This is (I suspect) because ripper was done "on the cheap" by reusing the parse.y file and injecting different terminal actions using #ifdef's.

Contrast your below output to the output from ruby_parser:

% rake debug R="x 1; x(1)"
s(:block,
  s(:call, nil, :x, s(:lit, 1)),
  s(:call, nil, :x, s(:lit, 1)))

since there's no semantic difference between the two, there shouldn't be a structural difference between the two. 

BTW, this output is for ruby_parser 3.0, which has not been released yet. I've gotten rid of an :args node and just splatted them into the :call node instead. RP 2 would output:

s(:call, nil, :x, s(:args, s(:lit, 1)))

RP3 release is coming "RSN".

> I suspect I may have more questions. Are there any good resources for
> Ripper, or anyone specific I should be pestering? I'm happy to put some
> documentation together as I explore further if people feel that would be
> useful?

I'm one of your better bets. You can probably cull a lot of your own answers from sexp_processor's pt_testcase.rb, which are the shared parser tests that I use across ParseTree, ruby_parser, ruby2c, ruby2ruby, and many others.

> Sample code and my output follow.
> 
> Thanks for your time.
> 
> -----
> 
> I'm running:
> 
> content = <<-EOS
>    x(1)
>    y 1
> EOS
> Sexp.from_array(Ripper::SexpBuilder.new(content).parse)[1]

PLEASE note that this is not a good idea... at least, if you're intending on passing said sexp through SexpProcessor. Problematic nodes shown below:

> The sexp I get out is
> s(:stmts_add,
>  s(:stmts_add,
>    s(:stmts_new),
>    s(:method_add_arg,
>      s(:fcall,
>        s(:@ident, "x", s(1, 0))

:@ident is not a processable node type, specifically because you can't have a method named "process_@ident" (at least... not easily).

s(1, 0) will bork for similar reasons, so you'll need the enclosing node to process it.

>      ),
>      s(:arg_paren,
>        s(:args_add_block,
>          s(:args_add,
>            s(:args_new),
>            s(:@int, "1", s(1, 2))

ditto on both cases

>          ), false
>        )
>      )
>    )
>  ),
>  s(:command,
>    s(:@ident, "y", s(2, 0)),

noticing a pattern?

>    s(:args_add_block,
>      s(:args_add,
>        s(:args_new),
>        s(:@int, "1", s(2, 2))
>      ), false
>    )
>  )
> )

It is for this reason that I haven't bothered trying to bridge ripper to my unified sexp format... it's just a mess.