From: Eric Mahurin <eric.mahurin@...>
Date: 2008-02-10T00:45:25+09:00
Subject: Re: Parsing JSON (#155)

------=_Part_31393_10119589.1202571932482
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Feb 9, 2008 12:59 AM, ThoML <micathom@gmail.com> wrote:

> > I was frankly amazed how much of the discussion was about speed.
>
> I personally found it quite interesting to see how well a hand-crafted
> parser could perform. I initially assumed the hackish RE-based
> solution would be fastest.
>

I would have thought that too.  I knew my LL(1) hand solution would be near
the top but was surprised that it came out on top (at least on my
machine/benchmark) especially considering the RE+eval (especially yours)
solutions.  I think object creation was a big factor.  My LL(1) parser
created basically nothing but the AST.  The match objects likely slowed down
the RE solutions (StringScanner was probably best off since the match object
is just a string).  When you turn off GC, some of the RE solutions beat
mine.

Another aspect would of course be how long it takes to create such a
> parser in comparison to the other solutions. Unfortunately, we don't
> have timings for that.
>

I'm guessing that you could crank out an initial RE solution the fastest.
But, those solutions were also the ones that failed as more tests were
added, since they were hacky.  The solutions that took no shortcuts and
followed the spec to a T from the beginning may have taken longer initially,
but they had fewer issues.


>
> > That said, Treetop is very slow, and we need to improve that.
>
> My main concern with treetop isn't so much speed but rather that the
> polygot approach. While the idea per se is pretty cool, it seems to
> preclude a programmatic generation/extension of a grammar definition.
> Eg store the rules as an array of lambdas, programmatically add a new
> rule on demand, recreate the parser etc.


My Grammar classes definitely give you that ability.  One of the many powers
of using Ruby as the DSL.

For example, if you deal with a "," separated list terminated by ";" for a
variety of items types, you could do something like this:

comma_list = lambda { |item|  item.list1(E(?;), E(?,)) }

then use it later like this

string_list = comma_list[string] # string defined earlier
number_list = comma_list[number] # number defined earlier

One of the powers is creating your own generic grammar constructs on the
fly.

------=_Part_31393_10119589.1202571932482--