From: Rick DeNatale Date: 2006-09-05T12:06:54+09:00 Subject: Re: Compiling Regexp only once So, I've corrected the benchmark. I've also added another method using String#split as Eric suggested. Here's the code being benchmarked: rick@frodo:~/rubyscripts$ cat stringsplit.rb class String To_chars_regex = Regexp.new('.') To_chars_regex2 = Regexp.new(/./) To_chars_regex3 = /./ def to_chars_array_with_unpack unpack('a'*length) end def to_chars_array_with_split split(//) end def to_chars_array_with_scan scan /./ end def to_chars_array_with_scan_precomp scan To_chars_regex end def to_chars_array_with_scan_precomp2 scan To_chars_regex2 end def to_chars_array_with_scan_precomp3 scan To_chars_regex3 end end Note that I made tested three different "pre-compiled" regex's one with a Regex.new('.'), one with Regex.new(/./), and one a constant with the literal regex /./. Now here's the benchmark: rick@frodo:~/rubyscripts$ cat benchstringsplit.rb require 'benchmark' include Benchmark load 'stringsplit.rb' iters = 100 bmbm do | x | str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("unpack #{str.length} character string") do iters.times do str.to_chars_array_with_unpack end end str += str end str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("scan-precomp #{str.length} character string") do iters.times do str.to_chars_array_with_scan_precomp end end str += str end str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("scan-precomp2 #{str.length} character string") do iters.times do str.to_chars_array_with_scan_precomp2 end end str += str end str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("scan-precomp3 #{str.length} character string") do iters.times do str.to_chars_array_with_scan_precomp3 end end str += str end str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("scan #{str.length} character string") do iters.times do str.to_chars_array_with_scan end end str += str end str = "abcdefghijklmnopqrstuvwxyz" * 5 5.times do x.report("split #{str.length} character string") do iters.times do str.to_chars_array_with_split end end str += str end end And here are the results: rick@frodo:~/rubyscripts$ ruby benchstringsplit.rb Rehearsal ----------------------------------------------------------------------- unpack 130 character string 0.950000 0.000000 0.950000 ( 1.225834) unpack 260 character string 0.920000 0.000000 0.920000 ( 1.736296) unpack 520 character string 0.950000 0.010000 0.960000 ( 2.474714) unpack 1040 character string 1.000000 0.000000 1.000000 ( 2.142817) unpack 2080 character string 0.940000 0.000000 0.940000 ( 2.772912) scan-precomp 130 character string 2.210000 0.000000 2.210000 ( 5.032766) scan-precomp 260 character string 2.190000 0.000000 2.190000 ( 7.615967) scan-precomp 520 character string 2.160000 0.010000 2.170000 ( 5.815543) scan-precomp 1040 character string 2.110000 0.000000 2.110000 ( 5.582919) scan-precomp 2080 character string 2.220000 0.000000 2.220000 ( 4.290547) scan-precomp2 130 character string 2.190000 0.020000 2.210000 ( 4.990034) scan-precomp2 260 character string 2.160000 0.020000 2.180000 ( 4.715208) scan-precomp2 520 character string 2.150000 0.040000 2.190000 ( 3.891652) scan-precomp2 1040 character string 2.180000 0.010000 2.190000 ( 3.757304) scan-precomp2 2080 character string 2.130000 0.010000 2.140000 ( 3.308557) scan-precomp3 130 character string 2.100000 0.000000 2.100000 ( 2.134875) scan-precomp3 260 character string 2.210000 0.040000 2.250000 ( 2.793601) scan-precomp3 520 character string 2.140000 0.000000 2.140000 ( 2.504348) scan-precomp3 1040 character string 2.090000 0.040000 2.130000 ( 2.872274) scan-precomp3 2080 character string 2.040000 0.000000 2.040000 ( 2.113700) scan 130 character string 2.240000 0.020000 2.260000 ( 3.108414) scan 260 character string 2.230000 0.010000 2.240000 ( 3.767771) scan 520 character string 2.170000 0.010000 2.180000 ( 3.892586) scan 1040 character string 2.280000 0.000000 2.280000 ( 2.540717) scan 2080 character string 2.190000 0.000000 2.190000 ( 2.454891) split 130 character string 2.350000 0.010000 2.360000 ( 2.749489) split 260 character string 2.330000 0.000000 2.330000 ( 2.620174) split 520 character string 2.370000 0.020000 2.390000 ( 3.186766) split 1040 character string 2.310000 0.040000 2.350000 ( 2.848300) split 2080 character string 2.460000 0.000000 2.460000 ( 2.930465) ------------------------------------------------------------- total: 60.280000sec user system total real unpack 130 character string 1.280000 0.040000 1.320000 ( 1.665192) unpack 260 character string 1.340000 0.040000 1.380000 ( 1.492881) unpack 520 character string 1.170000 0.010000 1.180000 ( 1.204762) unpack 1040 character string 1.190000 0.000000 1.190000 ( 1.855012) unpack 2080 character string 1.250000 0.000000 1.250000 ( 1.677772) scan-precomp 130 character string 2.500000 0.030000 2.530000 ( 3.720339) scan-precomp 260 character string 2.460000 0.010000 2.470000 ( 3.895554) scan-precomp 520 character string 2.470000 0.020000 2.490000 ( 3.777175) scan-precomp 1040 character string 2.430000 0.050000 2.480000 ( 2.955399) scan-precomp 2080 character string 2.520000 0.000000 2.520000 ( 2.828614) scan-precomp2 130 character string 2.330000 0.010000 2.340000 ( 2.668416) scan-precomp2 260 character string 2.280000 0.000000 2.280000 ( 2.322685) scan-precomp2 520 character string 2.310000 0.000000 2.310000 ( 2.335216) scan-precomp2 1040 character string 2.300000 0.000000 2.300000 ( 2.331397) scan-precomp2 2080 character string 2.370000 0.010000 2.380000 ( 2.477581) scan-precomp3 130 character string 2.400000 0.000000 2.400000 ( 2.456566) scan-precomp3 260 character string 2.340000 0.010000 2.350000 ( 2.416143) scan-precomp3 520 character string 2.550000 0.010000 2.560000 ( 3.359391) scan-precomp3 1040 character string 2.350000 0.040000 2.390000 ( 2.822663) scan-precomp3 2080 character string 2.350000 0.010000 2.360000 ( 2.649131) scan 130 character string 2.550000 0.050000 2.600000 ( 3.186714) scan 260 character string 2.490000 0.040000 2.530000 ( 3.098105) scan 520 character string 2.370000 0.000000 2.370000 ( 2.442281) scan 1040 character string 2.330000 0.000000 2.330000 ( 2.593032) scan 2080 character string 2.360000 0.010000 2.370000 ( 3.467826) split 130 character string 2.610000 0.030000 2.640000 ( 5.929269) split 260 character string 2.730000 0.040000 2.770000 ( 3.155784) split 520 character string 2.670000 0.030000 2.700000 ( 3.625916) split 1040 character string 2.730000 0.020000 2.750000 ( 8.130285) split 2080 character string 2.750000 0.010000 2.760000 ( 6.109507) So there are slight differences between using: PCR1 - A constant set to Regex('.') PCR2 - A constant set to Regex(/./) PCR3 A constant set to /./ and LIT a literal in-line /./ For this benchmark, for each string length tested, the techniques from fastest to slowest are; 130 Characters: Rehearsal: PCR3(2.10), PCR1(2.21), PCR2(2.21), LIT(2.26) "Live": PCR2(2.34), PCR3(2.40), PCR1(2.53), LIT(2.60) 260 Characters: Rehearsal: PCR2(2.18), PCR1(2.19), PCR3(2.25), LIT(2.25) "Live": PCR2(2.28), PCR3(2.40), PCR1(2.47), LIT(2.60) 520 Characters: Rehearsal: PCR3(2.14), PCR1(2.17), LIT(2.18), PCR2(2.19) "Live": PCR2(2.31), LIT(2.37), PCR1(2.49), PCR3(2.56) 1040 Characters: Rehearsal: PCR1(2.11), PCR3(2.13), PCR2(2.19), LIT(2.28) "Live": PCR2(2.30), LIT(2.33), PCR3(2.39), PCR1(2.48) 2080 Characters: Rehearsal: PCR3(2.04), PCR2(2.14), LIT(2.19), PCR1(2.22) "Live": PCR3(2.36), LIT(2.37), PCR2(2.38), PCR1(2.52) So, at least from this benchmark it doesn't seem that in-line literal regular expressions are faster than pre-compiled ones. In fact they never came in first, although PCR3 which was a constant set to a literal regex did win 4 times. Is this significant? Who knows, and with a new RegExp engine coming the numbers will be different in future. So as usual the right approach is test, profile to find out what needs improvement, and benchmark. -- Rick DeNatale My blog on Ruby http://talklikeaduck.denhaven2.com/