From: Arthur Chance <{spamtrap}@...> Date: 2003-04-17T03:14:12+09:00 Subject: Re: regular expressions "Hal E. Fulton" writes: > ----- Original Message ----- > From: "Arthur Chance" <{spamtrap}@qeng-ho.org> > Newsgroups: comp.lang.ruby > To: "ruby-talk ML" > Sent: Wednesday, April 16, 2003 8:12 AM > Subject: Re: regular expressions > [Me wittering about Snobol] > > [snip interesting SNOBOL stuff] > > First of all, it does my heart good to know that I am not > actually the oldest person on this list. When you were > programming in SNOBOL, I was barely in sixth grade, and > the only computers I knew of were in _Star Trek_ reruns. Excuse me while I go and quietly crumble in the corner. :-) As you get older either the memory goes or the mental arithmetic abilities, but I can't remember which. However, a recalculation shows me that it was only a mere 27-28 years ago that I started using Snobol. Even with that correction it was still back when I had far more hair and far less waist. :-) > More on-topic: I had an idea for "de-obfuscating" regexes > a year or more ago; and I may even have mentioned it on > the list. In fact, I think I did. Generally I wanted to > use a notation that was more like a programming language > (in a vein similar to the SNOBOL stuff you showed us) > which could (e.g.) be put into here-documents and compiled > into regular Ruby regexes. > > But eventually I was convinced that the Rockit parser could > do everything I wanted to do (though in a somewhat different > fashion, no doubt). So I abandoned that idea. > > I've never look at Rockit much. I assume it's in the RAA. > But I'd be curious to know what a SNOBOL hacker thinks > of it. I wouldn't exactly describe myself as a SNOBOL hacker (the main languages I used at the time were Algol68 + various assemblers), SNOBOL was just this great tool for mashing strings around, much like Ruby and Perl today. (Admission of dirty secrets: I used it mainly to take the fixed format output from Fortran programs and massage it into entirely differently shaped fixed format input for other Fortran programs. The people who I did it for thought it was miraculous, just the same as suits used to Office do today when watching an ad-hoc script in Ruby/Perl. Some things never change.) Anyway, I hadn't seen Rockit before you mentioned it, so this is an off the top of the head response. I have seriously used lexical and parser generators as well as hand writing lexers and parsers over the years (part of my career has been in academic CS, part designing and implementing proprietary languages for industry including one (Magik) that's remarkably like Ruby in feel. Matz and I obviously absorbed the same influences.) My general feeling is that regexps or similar ideas like Snobol patterns are at a different level from parsing. Of course, the fact that Rockit (like a few other parser generators) generates both lexer and scanner from the same input rather blurs the distinction, but I tend to think lexing is like brick laying and parsing is like architecture - you generally wouldn't want one job done by the practitioner of the other. The line from the SourceForge website > rockit-generated parsers builds the AST; NO need to write "action > code" in the grammar. "Action code" separated from grammar rather makes me wince. To be fair this may be because I grew up with LL(1) and LALR(1) based parser generators, where one often has to warp the grammar away from the ambiguous grammar that would produce a "natural" AST, and GLR parsing may not suffer this problem, but when working on compilers I want the AST I design, not one a program with no idea of what I'm going to do with it afterwards generates. (This could always be me just being an old fogey of course.) Generally I tend to believe in using tools that work for the job in hand but have never met one that does everything well. (I spent a couple of years working in PL/I which tried to be all things to all men. *Shudder*.) Rockit looks like it will be a useful tools for rapidly writing small and useful parser systems in Ruby, especially as it uses GLR, but it's overkill for a job regexps can already do. In truth the parsing problem was really solved for most practical purposes 10-20 years ago and another parser generator, even a GLR one is really a "me too" exercise. Now if someone can produce a Ruby framework for prototyping provably correct AST transformation, plus optimised code generation automatically derived from a machine specification, and spit it out in a form that can be used by C code, that would be wonderful. But that's a long way from regexp notation where we started. Maybe what's needed is Ruby code that takes a regexp denotation and explains it in English (or whatever human language is prefered), just like the program that turns C declarations into English. -- Conspiracy theories exist to prevent you knowing what's really going on.