[ruby-core:64467] [CommonRuby - Feature #10085] Add non-ASCII case conversion to String#upcase/downcase/swapcase/capitalize

Issue #10085 has been updated by Yukihiro Matsumoto.

Related to Feature #10152: String#strip doesn't remove non-breaking space added

----------------------------------------
Feature #10085: Add non-ASCII case conversion to String#upcase/downcase/swapcase/capitalize
https://bugs.ruby-lang.org/issues/10085#change-48413

* Author: Martin Dürst
* Status: Open
* Priority: Normal
* Assignee: Martin Dürst
* Category: 
* Target version: 
----------------------------------------
Case conversion functions are currently limited to ASCII characters. When used with formal languages, that may be appropriate, but it is often not appropriate for applications.

In order to avoid backwards-compatibility problems and to make sure that the various variants of case conversion (e.g. language-dependent) can be selected, we propose to add an optional parameter to the case conversion functions.

Our current design idea is as follows:

ASCII-only if no parameter:
'Türkiye'.upcase    # => 'TüRKIYE', note lower-case ü

Parameter triggers (general) Unicode conversion:
'Türkiye'.upcase 'en'  # => 'TÜRKIYE', note upper-case Ü

The parameter is actually a BCP 47 (http://tools.ietf.org/html/bcp47) language tag.
This means that for languages with special case conversion rules, such as Turkish, this works as follows:
'Türkiye'.upcase 'tr'  # => 'TÜRKİYE', note upper-case İ (with dot!)

In the second example, we used 'en', but most other languages would work, too, because a single case conversion works for most languages. Turkic languages are the biggest exception.

The Unicode standard also defines various cases of "case-folding", which usually is lossy, e.g. mapping German ß to
ss and so on. It should be possible to include this functionality in this proposal, e.g. by using :symbols or CONSTANTs for the few specific foldings. It may also be possible to define a reversible variant of case conversion in particular for use with swapcase.

In the long term, instead of a direct BCP 47 string, we could create a Locale class that would incorporate language-specific facilities, but this may need more detailed considerations.

The idea of using an additional parameter to indicate language-dependent or other processing variants should be extensible to areas such as number-to-string conversion and date formation. While this proposal is only about case conversion, we should check that there is a good chance to use similar parameter conventions for such extensions.

[This proposal is based on research done together with my student Kimihito Matsui.]


---Files--------------------------------
CaseConversion.pdf (340 KB)


-- 
https://bugs.ruby-lang.org/