From: Charles Hixson <charleshixson@...>
Date: 2000-11-19T21:10:02+09:00
Subject: [ruby-talk:6444] Ruby tokenizer for Ruby

Does anyone know of a Ruby tokenizer for Ruby?  In particular, I am bother 
by how to detect the boundaries of string constants and (to a much lesser 
extent) of regular expressions.  The only way I've been able to think of 
would involve multiple passes throught the code on a byte-by-byte level (in 
which case, Ruby is probably the wrong language to write it in).