[ruby-core:33014] [Ruby 1.9-Bug#4014] Case-Sensitivity of Property Names Depends on Regexp Encoding

Issue #4014 has been updated by Yui NARUSE.


Hmm, it's a difficult problem...

> run@paint:~$ ruby -e 'p /\p{ascii}/u'
> /\p{ascii}/
> run@paint:~$ ruby -e 'p /\p{ascii}/n'
> -e:1: invalid character property name {ascii}: /\p{ascii}/
> run@paint:~$ ruby -e 'p /\p{ASCII}/n'
> /\p{ASCII}/n
> run@paint:~$ ruby -e 'p /\p{ASCII}/u'
> /\p{ASCII}/

A spec may deny \p/\P for non Unicode regexps, it breaks some compatibility:
  Alnum, Alpha, Blank, Cntrl, Digit, Graph, Lower,
  Print, Punct, Space, Upper, XDigit, ASCII, Word
They are case sensitive and limited to \p/\P, not [[:alnum:]].
This has a good side effect that we can assume /\p{Alpha}/ must be a UTF-8 regexp.

Another spec may only allow lower case for non Unicode, but it seems late.
Martin says Unicode's guideline is wrong, but the compatibility for both ruby and other languages
following guideline seems correct.

RunPaint's suggestion is reasonable one, the patch is following:

diff --git a/regenc.c b/regenc.c
index b9b03b0..f0ddd2c 100644
--- a/regenc.c
+++ b/regenc.c
@@ -789,20 +789,20 @@ extern int
 onigenc_minimum_property_name_to_ctype(OnigEncoding enc, UChar* p, UChar* end)
 {
   static const PosixBracketEntryType PBS[] = {
-    PosixBracketEntryInit("Alnum",  ONIGENC_CTYPE_ALNUM),
-    PosixBracketEntryInit("Alpha",  ONIGENC_CTYPE_ALPHA),
-    PosixBracketEntryInit("Blank",  ONIGENC_CTYPE_BLANK),
-    PosixBracketEntryInit("Cntrl",  ONIGENC_CTYPE_CNTRL),
-    PosixBracketEntryInit("Digit",  ONIGENC_CTYPE_DIGIT),
-    PosixBracketEntryInit("Graph",  ONIGENC_CTYPE_GRAPH),
-    PosixBracketEntryInit("Lower",  ONIGENC_CTYPE_LOWER),
-    PosixBracketEntryInit("Print",  ONIGENC_CTYPE_PRINT),
-    PosixBracketEntryInit("Punct",  ONIGENC_CTYPE_PUNCT),
-    PosixBracketEntryInit("Space",  ONIGENC_CTYPE_SPACE),
-    PosixBracketEntryInit("Upper",  ONIGENC_CTYPE_UPPER),
-    PosixBracketEntryInit("XDigit", ONIGENC_CTYPE_XDIGIT),
-    PosixBracketEntryInit("ASCII",  ONIGENC_CTYPE_ASCII),
-    PosixBracketEntryInit("Word",   ONIGENC_CTYPE_WORD),
+    PosixBracketEntryInit("alnum",  ONIGENC_CTYPE_ALNUM),
+    PosixBracketEntryInit("alpha",  ONIGENC_CTYPE_ALPHA),
+    PosixBracketEntryInit("blank",  ONIGENC_CTYPE_BLANK),
+    PosixBracketEntryInit("cntrl",  ONIGENC_CTYPE_CNTRL),
+    PosixBracketEntryInit("digit",  ONIGENC_CTYPE_DIGIT),
+    PosixBracketEntryInit("graph",  ONIGENC_CTYPE_GRAPH),
+    PosixBracketEntryInit("lower",  ONIGENC_CTYPE_LOWER),
+    PosixBracketEntryInit("print",  ONIGENC_CTYPE_PRINT),
+    PosixBracketEntryInit("punct",  ONIGENC_CTYPE_PUNCT),
+    PosixBracketEntryInit("space",  ONIGENC_CTYPE_SPACE),
+    PosixBracketEntryInit("upper",  ONIGENC_CTYPE_UPPER),
+    PosixBracketEntryInit("xdigit", ONIGENC_CTYPE_XDIGIT),
+    PosixBracketEntryInit("ascii",  ONIGENC_CTYPE_ASCII),
+    PosixBracketEntryInit("word",   ONIGENC_CTYPE_WORD),
   };

   const PosixBracketEntryType *pb, *pbe;
@@ -811,7 +811,7 @@ onigenc_minimum_property_name_to_ctype(OnigEncoding enc, UChar* p, UChar* end)
   len = onigenc_strlen(enc, p, end);
   for (pbe = (pb = PBS) + sizeof(PBS)/sizeof(PBS[0]); pb < pbe; ++pb) {
     if (len == pb->len &&
-        onigenc_with_ascii_strncmp(enc, p, end, pb->name, pb->len) == 0)
+        STRNCASECMP(p, pb->name, pb->len) == 0)
       return pb->ctype;
   }

----------------------------------------
http://redmine.ruby-lang.org/issues/show/4014

----------------------------------------
http://redmine.ruby-lang.org