[ruby-core:54843] Re: [CommonRuby - Feature #7845] Strip doesn't handle unicode space characters in ruby 1.9.2 & 1.9.3 (does in 1.9.1)

2013/5/7 matz (Yukihiro Matsumoto) <[e-mail]>:
>
> But it's hard for me to imagine concrete problematic cases.
> When text from network connection is marked as Unicode, that's OK to process them as Unicode text,
> otherwise they should be marked as 'ASCII-8BIT' so that #strip and other methods should behave as
> they are now.

I see.  It seems less harmful than I expected.

However the encoding of a string can be easily changed if it contains
ASCII character only.

% ruby -e 'p(("a".force_encoding("UTF-8") +
"b".force_encoding("ASCII-8BIT")).encoding)'
#<Encoding:UTF-8>

So I think your sentence is bit weak to preserve current network libraries.
Appropriate restriction is "bahavior of methods for ASCII only strings
shold behave as the are now regardless of its encoding (ASCII-8BIT, UTF-8,
etc)".

(I assume the ASCII based network protocols, not Unicode based such as
naruse-san pointed.)

The famous example of such dangerous Unicode behavior is turkish case
conversion but it is locale dependent and you already said locale dependent
behavior is not target of this change.

I'm not sure that there are no such (locale-independent but affects ASCII
only string) Unicode behaviors, though.
Any experts here?
-- 
Tanaka Akira