From: Sean Russell <ser@...>
Date: 2002-05-31T09:25:28+09:00
Subject: Strange Ruby Regexp bug

This is really bizarre, but it isn't the first time I've seen it.  It is 
probably going to be impossible to track down.

I've got the following bit of code in REXML:

          nregexp = /^\s*<\/#{name}\s*>/mu
          puts nregexp.inspect

This is the /only/ puts I have in the program; IE, if I remove this puts, 
nothing is printed.  During the course of parsing a bunch of stuff, at some 
point, this gets printed:

        /(\340\023\010\260\3724\010nk\s*>/mu

How is the first regexp getting turned into the second regexp?  There's no 
parens anywhere in the original, and I guarantee that there are no parens 
in the value of 'name', unless something in the Ruby interpreter is putting 
them there.

Here's some interesting information

0) When I first encountered this, the exception was something completely 
different.  I went for a glass of water, and when I came back, it had 
magically changed into a different exception.
1) If I run the code like this, everything works, and all of the unit tests 
pass -- just a bunch of regexps are printed.  However, if I take the 'puts' 
line /out/, Ruby throws an exception:
  Error occurred in test_stream(Tester): RegexpError: unmatched (:
  /(\010x:\010de\s*>/
        ./rexml/element.rb:644:in `parse_stream'
  ...
2) If I take the nregexp line out, and embed the regexp directly in the next 
line that uses it, I get a completely /different/ regular expression:
  Error occurred in test_stream(Tester): RegexpError: unmatched (:
  /(Hx:\010de\s*>/
3) If I insert a comment (# ...) on the line /before/ the regexp is used, I 
get a /different/ regular expression:
  Error occurred in test_stream(Tester): RegexpError: unmatched (:
  /(\260w:\010de\s*>/

I have seen this before.  I can fix it by replacing the shorthand /.../mu 
with Regexp.new( s, Regexp::MULTILINE, 'u' ).  This appears to be a parser 
bug.  The really bizarre thing is that it appears to be caused by some 
bizare combination of the source file AND the input that is being parsed -- 
IE, I can run this same method a hundred times and have no problem, but 
then pass it a slightly different file to parse and the problem pops up.

I can /not/ cause this to reliably occur.  If I slightly change anything, it 
could morph into a different exception, or completely disappear.

Is this a known 1.6.7 bug, or is this new?  I'm running on Linux.

If there's anything I can do to help debug this, Matz, let me know.  This is 
probably going to be impossible to track down, though, unless you somehow 
have an insight on what could be causing this by the limited information 
I've provided :-/

-- 
 |..  I/O, I/O, it's off to disk I go, a bit or byte to read or write,
<|>    I/O, I/O.
/|\   
/|    
 |