From: Clifford Heath <no@...>
Date: 2008-02-04T14:45:00+09:00
Subject: Re: [QUIZ] Parsing JSON (#155)

Eric Mahurin wrote:
> I'm biased,

First up, it sounds like your package is a significant achievement,
and I don't want anything here to imply I'm not commending you on
it. But different approaches suit different goals...

>  but I don't think Treetop is super-clean and pure.  Take a look
> at my rubyforge grammar package if you want something really clean and pure:

:-) To me, to use a separate grammar is *more pure* than to use
a Ruby DSL. I can see how you would argue the opposite however...
Your example parsers are much harder to read than Treetop however,
compare:
	space = Grammar::Element[Set[?\ ,?\t].duck!(:==,:include?)].discard
with
	skip space
	  [ \t]
	end

(ok, so TT doesn't have a skip keyword yet, soon...)
That's what I mean by clean and pure anyhow. Not pure Ruby,
but pure grammar.

Nathan's concept is that "grammar" should become a Ruby keyword,
and should introduce the grammar meta-grammar, rather than using
existing Ruby syntax. I think that polyglot approach is much better
than the DSL approach, since the Ruby grammar isn't well-suited to
many important languages.

The point is, if/when Ruby uses a parser whose grammar is composable
with a DSL grammar you define, the result is truly seamless and
the syntax is unrestrained. Existing DSL syntax imposes too many
restraints. A language with a composable grammar would be truly
excellent!
 
> * specify language grammars in a BNF-like ruby DSL (no new grammar language
> to deal with!)

I think I've argued why that is a bad thing :-), though it definitely
has good things about it also - like your filters etc...

Ruby DSL is far too awkward to express CQL - which is my primary
project at present.

>   - integration of lexer/parser with other code is seamless since everything
> is ruby

Nice, but it won't help me when I need to target C# or Java.
Most important grammars need to be compilable to more than
one target language. Treetop's not there yet, but it's easy
to see how to do it, whereas it won't ever happen with yours
AFAICS.

>   - (sub)grammars are objects and you combine them using operators and
> methods

Can't get much easier than just "include otherGrammar", which
works in Treetop (mix the generated modules together).

> * complete API can be described in only a little more than a hundred lines

Similarly to Treetop, AFAICS.

> * infinite backtracking (not by default because of performance and
> functionality, you specify what needs it)

Does this memoize, or give exponential performance? Or is it optional?

> * can make any kind of pipeline of lexers/parsers and multi-thread them.
> * even with its pure ruby/no-regexp lexers/parsers, it can go head-to-head
> against Regexp-based stuff.  The flexibility of Regexp is limited, so I
> don't see the point since 'grammar' gets enough performance.
> * don't need to read the file into a string to parse it
> * and more

Very cool stuff.

> With  'grammar' you have complete control of  the  parsing result.  Without
> actions, the  output stream is a copy of the input stream.  You can group
> things and discard things efficiently.  One extreme would be to interpret
> the input on the fly and not build anything.

Yes, this is excellent, and something I've always wanted from a parser
generator. You should add your json parsers to the examples in the gem!

Clifford Heath.