From: Christian Neukirchen <chneukirchen@...>
Date: 2005-11-04T04:36:13+09:00
Subject: Re: YACC (Was: Re: ruby-dev summary 27393-27541)

MenTaLguY <mental@rydia.net> writes:

> On Thu, 2005-11-03 at 22:13 +0900, Christian Neukirchen wrote:
>> I don't think the real problem is the real grammar, that part of
>> parse.y looks like the easier one (and rather readable) to me.  The
>> problem is the lexer-parser communication, think heredocs, %q[] etc.
>> There is no way to express that in BNF.
>
> Well, you could, it would just be prohibitively verbose (well,
> infinitely long, if you don't place a limit on the number of characters
> in heredoc tags -- the special quote syntax by itself isn't so bad).
>
> However, I think we're still in the world of context-free grammars here
> -- heredoc/quote processing can still be handled by a pushdown
> automaton.  So it should be describable by a simple EBNF extension.
>
> Notionally, there would be a stack of "quote terminator" tokens, magic
> tokens like e.g. magic QUOTE_START and HEREDOC_START tokens which, if
> successfully matched, push the associated closing token onto the stack,
> and a QUOTE_END token which matches the closing token on the top of the
> stack and pops it if matched.

In this case, it may make sense not to use EBNF but some kind of
parser combinator?

(E.g. see http://chneukirchen.org/blog/archive/2005/10/the-rightyear-parsing-library.html )

This just would be needed to make language-independent enough to be
useful to all parties.

> Any other icky features of the grammar this wouldn't cover?
>
> -mental
-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org