From: Run Paint Run Run <redmine@...>
Date: 2010-11-06T04:24:49+09:00
Subject: [ruby-core:33076] [Ruby 1.9-Bug#4014] Case-Sensitivity of Property Names Depends on Regexp Encoding

Issue #4014 has been updated by Run Paint Run Run.


> - Only lowercase for everything. Explicitly diverge from Unicode TR#18 
> for the sake of Ruby-internal consistency. But we already allow 
> upper-case, so this would create a compatibility problem for Ruby.

Breaking backward-compatibility is unreasonable and unnecessary.

> - Allow variants for Unicode, only lowercase for non-Unicode. Each 
> follows tradition/specs, but the difference may be annoying, and there 
> at least should be some clear documentation.

This would break backward-compatibility as well, as the /\p{Katakana}/s example showed.

> - Allow variants for all encodings. Ruby will be more consistent 
> internally, but may not be consistent anymore with other non-Unicode 
> implementations.

By "non-Unicode implementations" are you referring to other regexp engines that don't support Unicode? If such beasts exist, is it our desire to be compatible with them?

> As it is easier to move from lowercase only to allowing variants than 
> the other way round, I think we should make sure a few more people think 
> carefully about this before we apply the patch below. 

It is confusing to raise a SyntaxError when the encoding of a regexp changes implicitly. This needs fixing either by ignoring property case for Unicode properties regardless of the regexp's encoding, or making all property names case-insensitive.

> Actually, looking at it, it doesn't accept things such as:
> ALNUM, ALnum, aLNUM, aLnUm,...
> As far as I understand Unicode TR#, these are all included in "whereby 
> the case distinctions, ... are ignored.".

My understanding is the same, and the patch works:

  run@paint-desk:~/mir/ruby$ patch <reg.patch 
  patching file regenc.c
  ...
  run@paint-desk:~/mir/ruby$ irb
  >> /\p{AlNUm}/  #=> /p{AlNUm}/
  >> /\p{AlNUm}/n #=> /p{AlNUm}/n
  >> /\p{AlNUM}/n #=> /p{AlNUM}/n
----------------------------------------
http://redmine.ruby-lang.org/issues/show/4014

----------------------------------------
http://redmine.ruby-lang.org