[ruby-core:33065] Re: [Ruby 1.9-Bug#4014] Case-Sensitivity of Property Names Depends on Regexp Encoding

On 2010/11/03 3:41, Yui NARUSE wrote:
> Issue #4014 has been updated by Yui NARUSE.

> Hmm, it's a difficult problem...

> Another spec may only allow lower case for non Unicode, but it seems late.
> Martin says Unicode's guideline is wrong,

Well, "was a mistake" is the better way to say it. It's the way it is 
now, so we have to live with it.

> but the compatibility for both ruby and other languages
> following guideline seems correct.

Well, there are essentially three choices:

- Only lowercase for everything. Explicitly diverge from Unicode TR#18 
for the sake of Ruby-internal consistency. But we already allow 
upper-case, so this would create a compatibility problem for Ruby.

- Allow variants for Unicode, only lowercase for non-Unicode. Each 
follows tradition/specs, but the difference may be annoying, and there 
at least should be some clear documentation.

- Allow variants for all encodings. Ruby will be more consistent 
internally, but may not be consistent anymore with other non-Unicode 
implementations.

As it is easier to move from lowercase only to allowing variants than 
the other way round, I think we should make sure a few more people think 
carefully about this before we apply the patch below. Actually, looking 
at it, it doesn't accept things such as:
ALNUM, ALnum, aLNUM, aLnUm,...
As far as I understand Unicode TR#, these are all included in "whereby 
the case distinctions, ... are ignored.".

Regards,   Martin.


> RunPaint's suggestion is reasonable one, the patch is following:
>
> diff --git a/regenc.c b/regenc.c
> index b9b03b0..f0ddd2c 100644
> --- a/regenc.c
> +++ b/regenc.c
> @@ -789,20 +789,20 @@ extern int
>   onigenc_minimum_property_name_to_ctype(OnigEncoding enc, UChar* p, UChar* end)
>   {
>     static const PosixBracketEntryType PBS[] = {
> -    PosixBracketEntryInit("Alnum",  ONIGENC_CTYPE_ALNUM),
> -    PosixBracketEntryInit("Alpha",  ONIGENC_CTYPE_ALPHA),
> -    PosixBracketEntryInit("Blank",  ONIGENC_CTYPE_BLANK),
> -    PosixBracketEntryInit("Cntrl",  ONIGENC_CTYPE_CNTRL),
> -    PosixBracketEntryInit("Digit",  ONIGENC_CTYPE_DIGIT),
> -    PosixBracketEntryInit("Graph",  ONIGENC_CTYPE_GRAPH),
> -    PosixBracketEntryInit("Lower",  ONIGENC_CTYPE_LOWER),
> -    PosixBracketEntryInit("Print",  ONIGENC_CTYPE_PRINT),
> -    PosixBracketEntryInit("Punct",  ONIGENC_CTYPE_PUNCT),
> -    PosixBracketEntryInit("Space",  ONIGENC_CTYPE_SPACE),
> -    PosixBracketEntryInit("Upper",  ONIGENC_CTYPE_UPPER),
> -    PosixBracketEntryInit("XDigit", ONIGENC_CTYPE_XDIGIT),
> -    PosixBracketEntryInit("ASCII",  ONIGENC_CTYPE_ASCII),
> -    PosixBracketEntryInit("Word",   ONIGENC_CTYPE_WORD),
> +    PosixBracketEntryInit("alnum",  ONIGENC_CTYPE_ALNUM),
> +    PosixBracketEntryInit("alpha",  ONIGENC_CTYPE_ALPHA),
> +    PosixBracketEntryInit("blank",  ONIGENC_CTYPE_BLANK),
> +    PosixBracketEntryInit("cntrl",  ONIGENC_CTYPE_CNTRL),
> +    PosixBracketEntryInit("digit",  ONIGENC_CTYPE_DIGIT),
> +    PosixBracketEntryInit("graph",  ONIGENC_CTYPE_GRAPH),
> +    PosixBracketEntryInit("lower",  ONIGENC_CTYPE_LOWER),
> +    PosixBracketEntryInit("print",  ONIGENC_CTYPE_PRINT),
> +    PosixBracketEntryInit("punct",  ONIGENC_CTYPE_PUNCT),
> +    PosixBracketEntryInit("space",  ONIGENC_CTYPE_SPACE),
> +    PosixBracketEntryInit("upper",  ONIGENC_CTYPE_UPPER),
> +    PosixBracketEntryInit("xdigit", ONIGENC_CTYPE_XDIGIT),
> +    PosixBracketEntryInit("ascii",  ONIGENC_CTYPE_ASCII),
> +    PosixBracketEntryInit("word",   ONIGENC_CTYPE_WORD),
>     };
>
>     const PosixBracketEntryType *pb, *pbe;
> @@ -811,7 +811,7 @@ onigenc_minimum_property_name_to_ctype(OnigEncoding enc, UChar* p, UChar* end)
>     len = onigenc_strlen(enc, p, end);
>     for (pbe = (pb = PBS) + sizeof(PBS)/sizeof(PBS[0]); pb<  pbe; ++pb) {
>       if (len == pb->len&&
> -        onigenc_with_ascii_strncmp(enc, p, end, pb->name, pb->len) == 0)
> +        STRNCASECMP(p, pb->name, pb->len) == 0)
>         return pb->ctype;
>     }
>
> ----------------------------------------
> http://redmine.ruby-lang.org/issues/show/4014
>
> ----------------------------------------
> http://redmine.ruby-lang.org
>
>

-- 
#-# Martin J. D端rst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[e-mail]