[ruby-core:25404] [Feature #2034] Consider the ICU Library for Improving and Expanding Unicode Support

Issue #2034 has been updated by Yui NARUSE.


> If ICU was not available could we not fall back to the current behaviour?

No, if so, it lose portability of scripts.
Codes will depend both Ruby's version and whether ICU is installed or not.
Codes wrote with ICU runs different in without ICU.
This will be a trouble maker.

> Well, String would be ideal.
> We would have the current function and a Unicode function,
> then choose between them when the method was invoked.

Adding Unicode sensitive functions to String may be accepted.
For example if the library is loaded, String#unicode_to_i is available.

> If that isn't possible, has the idea of String::Unicode been considered?
> I assume it would be too major of a change.

This seems hard to accept.
Ruby uses large class: don't split classes if they are similar.

> Another approach would be a 'unicode' library in stdlib that when loaded monkey-patched String.
> I don't favour this, however, because it seems hacky; String should work transparently with Unicode, IMO.

This is a approach 1.8's jcode.rb used.
Of course, it's not good.

> I suppose there are two distinct questions here:
>
> 1) Should String methods work correctly with Unicode by default?

NO.
Ruby's core methods treat ASCII as special.
For example, variables naming rule #1853, \s \w \d of regexp, and so on.

> 2) If so, do we hand-roll a solution or use ICU?

I won't write from scratch, but if Martin does, I don't object.
How about Martin?

> Perhaps the next stage is to investigate what methods in String need to be changed, and to what extent?

Matz said Ruby's core methods are ASCII sensitive, are't Unicode.
So exist methods won't change.
Unicode sensitive methods will be added as another methods.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2034

----------------------------------------
http://redmine.ruby-lang.org