From: James Gray <james@...>
Date: 2008-09-19T05:48:41+09:00
Subject: [ruby-core:18710] Encoding Safe Regexp.escape()

As part of my ongoing process to make CSV m17n savvy, I'm needing an  
encoding safe Regexp.escape().  I'm currently using basically this code:

   @re_esc   =   "\\".encode(@encoding)
   @re_chars =   %w[ \\ . [ ] ^ $ ?
                     *  + { } ( ) | ].map { |s| s.encode(@encoding) }

   # ...

   def escape_re(str)
     str.chars.map { |c| @re_chars.include?(c) ? @re_esc + c : c }.join
   end

First question:  are there logic flaws there or is escaping a string  
to safely use in a Regexp that simple?

I decided to check this and see how well it does on Ruby's built-in  
encodings.  I wrote and ran this simple script:

$ cat re_chars.rb
#!/usr/bin/env ruby -w
# encoding: UTF-8

Encoding.list.each do |encoding|
   begin
     esc = "\\".encode(encoding)
     re_chars = %w[ \\ . [ ] ^ $ ? * + { } ( ) | ].map { |s|  
s.encode(encoding) }
   rescue
     puts "Cannot convert to #{encoding}"
   end
end
$ ruby_dev re_chars.rb
Cannot convert to ISO-2022-JP-2
Cannot convert to UTF-7

It looks like I have at least two problem encodings.  I need to come  
up with a strategy for these.  Some options I am considering:

* Allow these to throw encoding errors and document that they are not  
supported.  I must admit that I've never encountered these encodings  
in the wild and I just suspect they may not be too common.  That leads  
me to question the effort spent on them when it seems I am  
successfully supporting 81 other encodings.  Of course, it would be  
even cooler to claim I support all of Ruby's encodings�

* Transcode incoming data for those two encodings to UTF-8.  I haven't  
checked if this is even possible, but I hope it would be for at least  
UTF-7.  This feels like giving up though.  I've come so far without  
resorting to this�

* Disable regular expression escaping for these encodings and hope for  
the best.  I doubt this will work though.  If I can't convert my ASCII  
regular expressions to the encoding, CSV is going to choke on it anyway.

Yeah, as you can see, I'm out of good ideas.  Please help.   
Suggestions very welcome.

One last question, any chance Ruby will eventually provide an encoding  
safe Regexp.escape() and allow me to skip this step?  :)

James Edward Gray II