From: MenTaLguY <mental@...>
Date: 2005-11-04T03:49:03+09:00
Subject: Re: YACC (Was: Re: ruby-dev summary 27393-27541)

On Thu, 2005-11-03 at 22:13 +0900, Christian Neukirchen wrote:
> I don't think the real problem is the real grammar, that part of
> parse.y looks like the easier one (and rather readable) to me.  The
> problem is the lexer-parser communication, think heredocs, %q[] etc.
> There is no way to express that in BNF.

Well, you could, it would just be prohibitively verbose (well,
infinitely long, if you don't place a limit on the number of characters
in heredoc tags -- the special quote syntax by itself isn't so bad).

However, I think we're still in the world of context-free grammars here
-- heredoc/quote processing can still be handled by a pushdown
automaton.  So it should be describable by a simple EBNF extension.

Notionally, there would be a stack of "quote terminator" tokens, magic
tokens like e.g. magic QUOTE_START and HEREDOC_START tokens which, if
successfully matched, push the associated closing token onto the stack,
and a QUOTE_END token which matches the closing token on the top of the
stack and pops it if matched.

Any other icky features of the grammar this wouldn't cover?

-mental