From: Eric Mahurin <eric.mahurin@...>
Date: 2008-02-04T13:53:55+09:00
Subject: Re: [QUIZ] Parsing JSON (#155)

------=_Part_4576_3702452.1202100841878
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Feb 3, 2008 7:44 PM, Clifford Heath <no@spam.please.net> wrote:

> Eric Mahurin wrote:
> > It was only about 6800
> > chars/second which is near the bottom of the heap.  Not sure how valid
> that
> > is since the parser has problems though.
>
> I'm not surprised. Treetop has been designed to be super-clean and pure,
> not fast. It'll get faster. Myself, I think it should have real Regex's at
> the leaves, and that'd make a *heap* of difference.


I'm biased, but I don't think Treetop is super-clean and pure.  Take a look
at my rubyforge grammar package if you want something really clean and pure:

* specify language grammars in a BNF-like ruby DSL (no new grammar language
to deal with!)
  - integration of lexer/parser with other code is seamless since everything
is ruby
  - (sub)grammars are objects and you combine them using operators and
methods
* complete API can be described in only a little more than a hundred lines
of code (but the implementation is a lot more since it has lots of
optimizations)
* heavily duck-typed such that a lexer or parser (or preprocessor, or parser
w/o lexer) can specified by the same DSL
* infinite backtracking (not by default because of performance and
functionality, you specify what needs it)
* can make any kind of pipeline of lexers/parsers and multi-thread them.
* even with its pure ruby/no-regexp lexers/parsers, it can go head-to-head
against Regexp-based stuff.  The flexibility of Regexp is limited, so I
don't see the point since 'grammar' gets enough performance.
* don't need to read the file into a string to parse it
* and more

Also, MRI is glacially slow at creating objects, which Treetop does a lot
> of (in this case, for every char in a string for example). If it used
> Structs,
> or if it used Rubinius, things would be a lot quicker.
>
> One other addition we're considering is to add skip rules, which don't
> build
> nodes, for things like white-space and punctuation, which would basically
> double the speed.


With  'grammar' you have complete control of  the  parsing result.  Without
actions, the  output stream is a copy of the input stream.  You can group
things and discard things efficiently.  One extreme would be to interpret
the input on the fly and not build anything.  I have an example tcl
interpreter done in a couple hundred lines of code using 'grammar', that
produces no AST - it just interprets while parsing.

------=_Part_4576_3702452.1202100841878--