From: Jeffrey Schwab <jeff@...>
Date: 2006-11-12T23:40:08+09:00
Subject: Re: regular expression too big

Peter Schrammel wrote:

> got  problem with big regexes:
> I have a regex of about 70000+ words concated with '|' that I'd like to
> match as a regex. /bla|blub|foo|bar|.....(70000)/
> 
> But unfortunately ruby gives me a 'regular expression too big' if I'm
> trying to build such a thing.
> I had a look at the regex.c code and saw the limit of 1 << 16 bytes for
> regexes. Is there a way around this (without going down to 2000 words) ?
> 
> Thanks for any hint

You could optimize the regex a little for size, e.g. by factoring out 
common prefixes:

	(b(l(a|ub)|ar)|foo)...


Of course, that will only help if the | alternatives have a reasonable 
amount of redundancy.  Alternatively, you could just break the whole 
thing into multiple expressions.  Instead of

	if /first_part|second_part/ =~ text

You could try:

	if /first_part/ =~ text or /second_part/ =~ text