From: Mike Harris <GENIE@...>
Date: 2006-06-27T00:35:31+09:00
Subject: Re: Regex Comparison Causing Massive Memory Usage

Xavier Noria wrote:

> On Jun 26, 2006, at 17:04, Mike Harris wrote:
>
>> Has anyone run into a problem where a regex comparison against a  
>> very large string causes memory usage to massively and rapidly  spike 
>> upwards.  A string of length ~200000 causes memory usage to  reach 
>> and exceed 2 gigs within seconds, crashing the running program.
>> I'm not looking to diagnose the problem (with this e-mail) or  
>> anything like that, which is why I haven't provided details.  Just  
>> curious if others have run into this problem.
>
>
> Some regexps explode because of a mixture of alternations,  
> quantifiers, etc. This is intrinsic to them so to speak, not  
> dependant on the engine. How does yours look?
>
> -- fxn
>
>
>
>
"My" Regex is not actually mine.  It's in Ruby's SOAP Implementation.  
The crash is (I believe) in this method

  def Charset.is_utf8(str)
    UTF8Regexp =~ str
  end

and the regex in question looks like this

  character_utf8 =
    "(?:#{us_ascii}|#{twobytes_utf8}|#{threebytes_utf8}|#{fourbytes_utf8})"
  UTF8Regexp = Regexp.new("\\A#{character_utf8}*\\z", nil, "NONE")