From: Daniel DeLorme <dan-ml@...42.com>
Date: 2007-12-05T11:03:15+09:00
Subject: Re: Unicode in Regex

Daniel DeLorme wrote:
> Heavy compared to what? Once compiled, regex are orders of magnitude 
> faster than jumping in and out of ruby interpreted code.

Sorry to beat a dead horse, but I just did an interesting little 
experiment with 1.9:

 >> str = "abcde"*1000
 >> str.encoding
=> <Encoding:ASCII-8BIT>
 >> Benchmark.measure{10.times{ 1000.times{|i|str[i]} }}.real
=> 0.010282039642334
 >> str.force_encoding 'utf-8'
 >> Benchmark.measure{10.times{ 1000.times{|i|str[i]} }}.real
=> 1.29934501647949
 >> arr = str.scan(/./u)
 >> Benchmark.measure{10.times{ 1000.times{|i|arr[i]} }}.real
=> 0.00343608856201172

indexing into UTF-8 strings is *expensive*

Daniel