From: Clifford Heath <no@...>
Date: 2008-02-09T20:24:55+09:00
Subject: Re: Parsing JSON (#155)

ThoML wrote:
>> I was frankly amazed how much of the discussion was about speed.
> I personally found it quite interesting to see how well a hand-crafted
> parser could perform. I initially assumed the hackish RE-based
> solution would be fastest.

Yes, it was interesting. I just thought that a few other things
were interesting too ;-)

> My main concern with treetop isn't so much speed but rather that the
> polygot approach.

Well, I'm the author of the Polyglot gem, and that doesn't preclude
the idea of executing Ruby blocks during the parse. I'd be much in
favor of that actually, but it's done Nathan's way as he created
Treetop. I like the way ANTLR and grammar let you build the parse
tree you want. Or not. But remember, packrat needss parse tree nodes
that contain enough to enable the backtracking.

Since Treetop has the ability to request a custom node (using the
<my_syntax_node_class> annotation) perhaps you can do what you want
by adding code to the constructor in that class? Just remember though,
the node might get discarded during backtracking, so don't do anything
too permanent :-).

> Eg store the rules as an array of lambdas, programmatically add a new
> rule on demand, recreate the parser etc. If I understand it right,
> this isn't easily possible with treetop.

I think it would be a sin against the gods of good taste to do that,
especially when the thing that did it might get discarded!

> I scanned the source code a
> little bit and it seems treetop requires the parser definition to be
> saved on disk. I assume you're now going to tell me I'm wrong? Please
> do so. I'd be glad to here it's otherwise.

It's otherwise. The compiler uses a Builder pattern, and then either
saves that or eval's it. When Polyglot requires a grammar, it first
passes through the normal (or rubygems+normal) require, which will
load a .rb file if one exists in the path. Otherwise it compiles
the grammar file and eval's it.

I think, without testing it, that multiple require's in the 2nd
situation will compile the grammar multiple times, which would be
a bug... I'll test for that tomorrow... bed now ;-).

Clifford Heath.