From: Robert Feldt <feldt@...>
Date: 2002-01-14T18:50:32+09:00
Subject: Re: Markus' ruby-parser

On Sun, 13 Jan 2002, Mathieu Bouchard wrote:

> Note: I didn't choose Rulator/Rockit because I've been told (by Robert)
> that it is still too experimental, but that was a while ago. If this is
> not true anymore I'd like to know. But there's also a reason I chose
> Markus' parser in particular... it is that it only supports MetaRuby's
> RubyAST.rb, and it is also the only parser to support it now. However our
> versions have forked and I want to merge them back together.
> 
> Note that I have the intention of bundling one parser with MetaRuby. I'd
> like to know which one would be the best, and why.
> 
Let me just summarize the state of Rulator as compared to markus parser:

* Lexer is pretty well tested but not all constructs in the language has
been tested with the parser so some bugs still there. What is needed is to
write some more tests for the parser (singletons for example).

* Lexer is written with Rockit's parser combinators while Markus directly
translated Matz lexer. So there might be more deviances in Rulators lexer
although its shorter and may be simpler to change. Currently its probably
slower though.

* Rulator does not use RubySchema but the similar but simpler Terms I use
in Rockit. The main difference is higher resolution (more constructs,
ie. "a+b" is Plus[Id["a"], Id["b"]] instead of Expr[Id["a"], :+,
Id["b"]]), no typing (which IMHO, is ok for parsers but may be needed when
used to manually construct Ruby code) and that Terms can be
pattern-matched (this is useful when writing translation rules ie
converting from one language to another). Differences are not large though
so we should probably work to merge the two approaches.

* The parsers calls a handler object instead of directly constructing the
AST. By plugging in your own handler object you can turn it into an
event-based parser (ie. the handler is the event handler and the default
handler builds an AST). So Rulator can use RubySchema if someone writes a
handler that builds RubySchema's instead of RubyTerm's. Calling the
handler buys flexibility for a small performance penalty.

* Comments are added as attributes to the RubyTerms. They're pre_comment
for coments directly preceding a language construct post_comment in the
symmetric way.

So I think a fair summary would be:
* Markus parser is a bit more tested.
* Rulator's lexer is a bit more tested but not as close to matz lexer as
markus. Its shorter though so might be simpler to debug/maintain.
* Rulator is a bit more flexible but slower.
* They use similar but slightly different representation of the
AST's. The comunity should probably settle for one solution here since it
can be used in many situations (not only parsers). Matju and I have
already discussed how the resolution of RubySchema can actually be
increased if one wants to (Plus = Expr.of(:+) for example). IMHO, that
might be needed to get pretty-prints close to original, simpler
specification of translations (example: I have different code to
execute based on the operator of an expression; I shouldn't need to
access and compare it again) etc. 

That's about it. Our intention has been to merge them but it hasn't
happened yet. My impetus for getting Rulator out slowed when Dave
found another solution for the RDoc parsing (and the deadline for my PhD
project started coming really close... ;-)).

Regards,

Robert