[ruby-core:25440] [Feature #2034] Consider the ICU Library for Improving and Expanding Unicode Support

Issue #2034 has been updated by Yui NARUSE.


> > Adding Unicode sensitive functions to String may be accepted.
> > For example if the library is loaded, String#unicode_to_i is available.
> 
> Do you like that API? It feels clumsy to me.

No, I think so too.

> Users will have to change their code just to handle Unicode strings, when
> Ruby could make the determination automatically.

Those Unicode sensitive methods leave as is for compatibility
even if Ruby could make the determination automatically.

> > Matz said Ruby's core methods are ASCII sensitive, are't Unicode.
> > So exist methods won't change.
> 
> :-( 

Ruby core methods are for programers.
So naming collisions are resolved for programers side.

> FWIW, Python uses .. as the API. IOW, a generalised approach of the Onigurma patch.

Thanks,

> It would take me a while, but if Martin doesn't have the time, I may be able to
> produce something along those lines, albeit without the clever optimisations initially.

If you do, I'll discuss about this and its API on next developper's meeting.

> Another data point: Perl 6 is optionally linking against ICU (http://github.com/rakudo/rakudo/tree).

Ruby hates compile options which change behaviors over Ruby Layer.
So those options are hard to be merged.

> I cannot speak for Ruby core team but I personally do not like ICU.
>
> It's not that I am against Unicode but Unicode is only part of what
> Ruby aims to support yet ICU is huge and written in C++ which would
> drastically enlarge Ruby core size and reduce portability.

Yes, so if we use ICU, the library will be a bundled or external library, not core.

> Better support for character classes outside of the ASCII range is
> certainly desirable but it requires more design and planning than
> "Hey, I saw ICU, it's cool, let's use it".

true.

> First the questions
> 
>  - What exactly we want supported for what purposes?

Unicode Normalization or other conversions, Unicode sensitive case convertions or matches
and so on are the base framework of current internet specs.
For exmaple, if uri library supports IRI, they need those functions.

>  - What the cost would be?

Human resource and distribution size.

>  - Do we really want to pay that cost?

YES, more and more specs depend on Unicode.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2034

----------------------------------------
http://redmine.ruby-lang.org