From: Clifford Heath <no@...>
Date: 2008-10-01T12:03:57+09:00
Subject: Re: my ears are burning... ;)

parrt@antlr.org wrote:
> Well, I wasn't talking about packets; more protocols and things like
> that.

Protocols are just a two-directional sequences of packets. They
always require parsing, though sometimes the hardware does some
of that (like Ethernet frames)
.
> Oh, right. Grimm/Rats! guy told me about that trick.  Nice, though I'm
> guessing you're going to have trouble with recovery. ;) Bolting on the
> first error is something most people don't want to happen.

Agree - but recovery isn't so hard. Just use a catch-all alternative
that gets activated when nothing else matches, and get it to skip to
a recovery point. The problem is you lose the error context from the
failure though. Perhaps the current "best guess" error list should be
available to the recovery rule... thanks for inspiring that idea, I
might be able to make it work.

> You bet. Incremental parsing with recursive descent parsers has to be
> done with a mechanism strikingly similar to memoization.  That's the
> easy part once you figure out the trick. the part that I have not
> figured out is the lexer.  Since PEGs are scannerless usually, perhaps
> the same mechanism just plain works; let me know how that goes. I
> would be interested in the results.

It should just work, though AIUI the invalidation rule are rather subtle.

> Actually, my experience is that people like ANTLR because they can put
> actions anywhere. Over 24 years  observing parser building, It seems
> that adding arbitrary actions to the grammar is pretty common.

Right, I can see that. In my CQL grammar, it's a sequence of isolated
definitions that may reference earlier definitions. Each definition is
quite difficult to parse, but the sentence structure is flat. I tell
Treetop to parse a definition and not worry about unconsumed input,
rinse and repeat. The only thing I really need semantic predicates for
is to detect when a symbol ref's an earlier definition, i.e. symbol-
table lookup, and I'm currently doing without that (so some features
of the grammar can't be implemented). However. that makes it pretty
easy to envisage PEG support for the kind of predicates I need.

>  Many
> users are uncomfortable building an intermediate data structure.

Most "Ruby DSLs" *are* just intermediate data structures (hashes, arrays
and symbols, perhaps yielded by code blocks, so most of us are pretty
comfortable with it... no need to create custom classes to represent
the intermediate tree as the builtin classes work just fine.

>> I'd like to find the
>> time to make ANTLR work properly with Ruby, but I already have
>> a (more) significant project under way. Maybe Eric will do it?
> 
> As before, I'm very happy to help. Minus the optional parentheses, I
> like Ruby :)

I got stuck with ANTLR's Ruby templates due to a lack of documentation
of what data was available to each template. I guess it's just a parse
tree representation of some sort, but I wasn't familiar enough with
ANTLR to figure it out. An ability to define a template as "Stub - print
a diagnostic telling me all the things available here" would have helped
a lot.

Clifford Heath.