From: "mame (Yusuke Endoh) via ruby-core" <ruby-core@...>
Date: 2026-11-02T13:15:04+00:00
Subject: [ruby-core:127296] [Ruby Bug#22493] Case-insensitive matching is inconsistent in some single-byte encodings

Issue #22493 has been reported by mame (Yusuke Endoh).

----------------------------------------
Bug #22493: Case-insensitive matching is inconsistent in some single-byte encodings
https://bugs.ruby-lang.org/issues/22493

* Author: mame (Yusuke Endoh)
* Status: Open
* Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN
----------------------------------------
Each single-byte encoding has two case tables: the lowercase table (used for matching and `String#downcase`) and the list of case pairs (used for the search optimization and character classes). They disagree in some encodings, so case-insensitive matching gives different results depending on the direction or the search method, and some characters that are not case pairs are matched with each other.

```ruby
e = Encoding::Windows_1251
big = "\x80".force_encoding(e)    # "��"
small = "\x90".force_encoding(e)  # "��"
Regexp.new(big, Regexp::IGNORECASE) =~ small             #=> 0
Regexp.new(small, Regexp::IGNORECASE) =~ big             #=> nil (expected: 0)
big.start_with?(Regexp.new(small, Regexp::IGNORECASE))   #=> true

e = Encoding::Windows_1252
Regexp.new("\xD7".force_encoding(e), Regexp::IGNORECASE) =~ "\xF7".force_encoding(e) #=> 0 (expected: nil; "��" and "��")
```

@make_now_just found this and wrote the fix. The fix makes the two tables agree, following the Unicode case mapping of each character. All the changes are:

* ISO-8859-16
  * lowercase table: 0xA4 ��� no longer maps to 0xA5 ���
* Windows-1250
  * lowercase table: 0xD7 �� no longer maps to 0xF7 ��
* Windows-1251
  * case pairs added: ��/�� (0x80/0x90), ��/�� (0x81/0x83), ��/�� (0x8A/0x9A), ��/�� (0x8C/0x9C), ��/�� (0x8D/0x9D), ��/�� (0x8E/0x9E), ��/�� (0x8F/0x9F), ��/�� (0xA1/0xA2), ��/�� (0xA3/0xBC), ��/�� (0xA5/0xB4), ��/�� (0xAA/0xBA), ��/�� (0xAF/0xBF), ��/�� (0xB2/0xB3), ��/�� (0xBD/0xBE) (they were already in the lowercase table)
* Windows-1252
  * lowercase table: 0xAF �� no longer maps to 0xBF ��, and 0xD7 �� no longer maps to 0xF7 ��
* Windows-1253
  * lowercase table: 0xB6 �� no longer maps to 0xDC ��
  * case pairs: ��/�� (0xB6/0xDC) replaced with ��/�� (0xA2/0xDC)
* Windows-1254
  * case pairs added: ��/�� (0x8A/0x9A), ��/�� (0x8C/0x9C), ��/�� (0x9F/0xFF) (they were already in the lowercase table)

The wrong entries seem to come from copying the tables of another encoding: windows_1252.c was made from windows_1251.c (0xAF/0xBF and 0xD7/0xF7 are case pairs in Windows-1251), Windows-1253 from ISO-8859-7 (where 0xB6 is ��), and Windows-1254 from ISO-8859-9 (which has no characters in 0x80-0x9F).

This does not change the Greek final sigma �� and ��, or the Turkish ��, which also differ from Unicode case folding.

@duerst @ima1zumi Could you review these changes?



-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/