From: Mark Probert <probertm@...>
Date: 2002-09-24T01:48:38+09:00
Subject: Lexing ... (was: Re: Regexp: How to Find Legitimate Tokens?)

At 11:44 PM 9/23/2002 +0900, Bill wrote:

>[snip]
>4) Are tools such as lex and yacc more powerful than Ruby in terms of
>their lexing and parsing capabilities?  Without regard to efficiency, is
>to use Ruby as easy as to use lex?  Is to use Ruby as easy as to use
>yacc?  (Probably no, such as shown by the discussion on a parser tool in
>Ruby?)

One data point.

Given the recent discussion on lexer/parsers for Ruby, I thought that I
would do something I have been meaning to do for a while, port Coco/R to
ruby.

For those who don't know, Coco/R is an LL(1) recursive decent scanner/parser
that basically takes an attributed EBNF as its input and generates the
scanner, parser and a driver.  The things that I like about Coco/R is that
the attributed grammar is easy to understand and everything is in one place
(no separate lex and yacc files) and that it is very fast.

So, I have taken the C/C++ source and started the convert-to-ruby process.
I have the table-driven scanner done and I have to say that ruby is -very-
powerful at doing this stuff.  Given that this is a first-pass, the code
is a little ugly and not optimised in -any- way, but it performs pretty
well.  The scanner is around 500 lines of commented code.  I expect that
to drop.  And I have found it easier to write than lex code.

Now on to the parser!  Then no more yacc-ing ...

-mark.