From: Ken Bloom <kbloom@...>
Date: 2008-10-13T23:07:12+09:00
Subject: Re: regexp with accent insensitive ??

On Sun, 12 Oct 2008 15:00:29 -0500, Davi Barbosa wrote:

> Hello,
> Is there anyway to make the regexp accent-insensitive? (/a/ match with ã
> and Ã)
> 
> If not, can any one give a solution to my problem: I'm making a search
> web page with mod_ruby, so I made an accent/case-insensitive sql query
> and this works fine (with latin1_swedish_ci). Now I want to highlight
> what the user searched for. To achieve this I'm doing something *like*:
> string.gsub(/search/i,'<span class="highlight">\0</span>') This works
> fine if search and the relevant part of string don't have accents, but
> if there are any accents it doesn't match, so the entry is not
> highlighted.
> 
> I know that with
> Iconv.conv("ascii//translit","UTF-8",str) I can remove all the accents
> from str, so I can remove the accents from 'search' without any problem,
> but if I remove some accents from string to do the highlighting, I need
> to put it back later to display it to the user.
> Does anyone have any idea?
> 
> Thank you

in which case, I would try replacing the accented letters with periods 
(which match any single character) when searching. This will give some 
false positives. I would use gsub with a block to do a more specific 
conditional test.

Suppose the search was for ole (without the accent, and the real hits 
will have an accent on the e) the search is in a language that allows 
accents on only the letter e.

query='ole'
pattern=Regexp.compile('ole'.gsub(/[e]/,'.')) #=> /ol./

translit=Iconv.conv("ascii//translit","UTF-8",'ole') #=> "ole"

gsub(pattern) do |match|
  #use the regular expression to get close enough, and to get
  #the actual text we're concerned about
  if Iconv.conv("ascii//translit","UTF-8",match) == translit
    #the if test does the actual exact comparison
    "<span class=\"highlight\">#{match}</span>"
  else
    match
  end
end

Of course, there may be some locale tricks that I'm missing that would 
make this much easier.

-- 
Chanoch (Ken) Bloom. PhD candidate. Linguistic Cognition Laboratory.
Department of Computer Science. Illinois Institute of Technology.
http://www.iit.edu/~kbloom1/