From: Eric Mahurin Date: 2008-09-24T21:23:23+09:00 Subject: Re: ANTLR Target for Ruby ------=_Part_58891_21525374.1222259012335 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Tue, Sep 23, 2008 at 9:04 PM, Clifford Heath wrote: > DSLs are fine for folk who don't know how to make proper grammars :-). > OK. I guess that's that source of the "bleeding eyes"... I'm sounds like I won't be able to convince you why a DSL is better than a specific format, and you vice-versa. A quick summary is: although you lose the exact syntax that you may want, you gain in power and flexibility. Using Grammar (directly), you could template Grammars (method/lambda could return a Grammar based on Grammar args and other params), use plain ruby for extra control when specifying a Grammar tree, or simply embed Grammar objects in a ruby program (like Regexp objects can be - although they parse the grammar from a string). I would like >> to provide translators from various formats (BNF, PEG, regex, etc) to >> Grammar objects. >> > > That's what I'm talking about, good stuff. > > Also, with Grammar 0.8, I've completely separated the parser generation. >> Grammar is just a light-weight layer used to hold the description of the >> language you are parsing. >> > > So you're already halfway to a VM, where the generated parser doesn't have > to be human-readable at all :-) Better - it is a compiler. Compiles from a Grammar to a target language. Grammar has always done this. Previously, Ruby code generation was done in the Grammar class itself. I've separated it to add quite a bit of flexibility. > > Personally, I don't like the idea of automatic backtracking. Regexp does >> this. The big problem I see with auto-backtracking is that when the parse >> fails, you just get a simple "it failed" response. >> > > Not the case with Treetop. It assumes that the furthest point reached is > the failure point (not often true but an assumption that works, given...) > and lists the text leading up to that, and enumerates the terminal tokens > that would have allowed it to get further. This works *really* well. > Try it - call the "failure_reason" method on a failed parse to see it > in action. > OK. I guess it just remembers the furthest point it reached when backtracking and when all possibilities are exhausted, it reports this furthest point. Sounds reasonable. I guess I could try something like this when I backtrack. Right now, I ignore errors/mismatches that cause a backtrack. > Does Treetop auto-backtrack? >> > > That's what memoizing is for - it's the core of how PEG works. > Don't you mean packrat instead of PEG? I thought PEG was just a format for describing something to parse, like BNF. And packrat refers to the type of parser - like LL or LALR. From my understanding packrat is a cousin of LL, but adds memoizing to achieve linear backtracking performance. > > If backtracking isn't always on, do you memoize only when you might >> possibly >> backtrack? In Grammar, I'm wondering if it is possible to take to a >> memoizing performance hit only when backtracking is allowed. >> > > I don't think you can guess when backtracking will be needed. > As I said, it requires heavy analysis - Terrence didn't get his > PhD for nothing! > In Grammar, I don't try to guess. Backtracking is normally not done. To allow for backtracking, I provide a Grammar#backtrack method that returns a new Grammar that can backtrack to the beginning of where the Grammar started parsing the input when it fails in the middle. I would only apply memoization for one of these - if this is doable. > I already found it on the antlr site, but not with a ruby target. I >> didn't >> want to go relearn antlr (v3 now) and how to get it to generate ruby. >> > > You just ask for Ruby in your options block at the top of the grammar. > The Ruby target is quite strange and limited though IIRC. The lexer > in all ANTLR grammars is definitely non-intuitive, it doesn't choose > the longest possible match like every other lexer in the world. Terr > isn't concerned about his users, just about big-noting himself. I > wouldn't say "their weak support ran me off" so much as just "they ran > me off". When he pisses someone off, that just proves his superiority > more... and I have the emails to prove it. You mighta thought DHH had > an attitude problem, heh, he's tame! > I didn't encounter that a few years ago. I mostly just learned a lot about LL parsing and wanted to make something better. It does sound like others have been run off though... > Have you used ANTRWorks BTW? It's excellent! >> Once. I'd like to make something like this for Grammar (at least an >> engine >> that graphically displays a Grammar). >> > > ANTLRWorks shows railroad diagrams, which I used to produce syntax > documentation for CQL. But the ability to interactively test individual > rules is great. A lot of work though - I'd have thought your skills > were better spent elsewhere. > I was only thinking making a Grammar engine that generated dot files for Graphviz to get a visualization of a Grammar. Eric ------=_Part_58891_21525374.1222259012335--