From: Eric Mahurin <eric.mahurin@...>
Date: 2008-02-06T10:42:15+09:00
Subject: Re: Parsing JSON (#155)

------=_Part_13445_33249281.1202262141875
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Feb 5, 2008 6:44 PM, Clifford Heath <no@spam.please.net> wrote:

> Eric Mahurin wrote:
> > With Regexp, you find it easiest
> > and safest to read the entire file/stream into a String first.  IMHO, a
> > "real" parser should not need to read the entire file into memory.
>
> Agreed. It should however be possible to modify a Regexp engine to
> work on an IO stream, reading more input only as required. Because
> of the lookahead required, it is necessary to be able to wind back
> excessive input that has been read.
>

When I first started writing my parser generator, I wanted it to be able to
handle a regex at the leaf.  I tried putting some layer on top, then I
complained about the inflexibility of regex.  Finally I gave up and decided
not to use them.  I think I ended up with something much cleaner since a
regex is already a mini-parser.  Using two parsing languages (low-level
regexp and high-level BNF-like thing) just wasn't as appealing anymore.
Fortunately, I found out much later that this decision didn't really cost
anything in terms of performance.

But, yes regex should be changed to be more flexible.  In my opinion, it
should be duck-typed to work on any String-like object (using a subset of
string methods - #[] would be minimal).  It's not to difficult to wrap an IO
to look like a read-only String.  It would of course optimize the real
String case, but that shouldn't effect the functionality.

Even C++ has "duck-typed" a regex:

http://www.boost.org/libs/regex/doc/index.html

This works on arbitrary characters and arbitrary (external) iterators (into
containers).  Of course it uses the ugly template syntax to map to static
types.

------=_Part_13445_33249281.1202262141875--