[ruby-core:53374] [ruby-trunk - Bug #7845] Strip doesn't handle unicode space characters in ruby 1.9.2 & 1.9.3 (does in 1.9.1)

Issue #7845 has been updated by threedaymonk (Paul Battley).


It's true that whitespace *can* be locale-dependent, at least insofar as Unix locales can specify which codepoints are to be considered as whitespace (in addition to space, tab, etc.).

The linked conversation about capitalisation doesn't seem relevant, though. The problem with capitalisation is that it really is language-specific: uppercase i is not I in Turkish, for example. Space, on the other hand isn't; it's just that the inventory of spaces used and their encoding depends on the locale: you won't find a double-width space in English, and the representation in UTF-8 and EUC-JP is different.

However, given that String#upcase/downcase are basically useless for non-ASCII content, I'm not sure I'd expect strip to handle Unicode spaces either.
----------------------------------------
Bug #7845: Strip doesn't handle unicode space characters in ruby 1.9.2 & 1.9.3 (does in 1.9.1)
https://bugs.ruby-lang.org/issues/7845#change-37573

Author: timothyg56 (Timothy Garnett)
Status: Rejected
Priority: Normal
Assignee: naruse (Yui NARUSE)
Category: M17N
Target version: 
ruby -v: ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-linux]


Strip and associated methods in ruby 1.9.2 and 1.9.3 do not remove leading/trailing unicode space characters (such as non-breaking space \u00A0 and ideographic space \u3000) unlike ruby 1.9.1.  I'd expect the 1.9.1 behavior.  Looking at the underlying native lstrip! and rstrip! methods it looks like this is because 1.9.1 uses rb_enc_isspace() whereas 1.9.2+ uses rb_isspace().

1.9.1p378 :001 > "\u3000\u00a0".strip
 => "" 

1.9.2p320 :001 > "\u3000\u00a0".strip
 => "　 "

1.9.3p286 :001 > "\u3000\u00a0".strip
 => "　 "


-- 
http://bugs.ruby-lang.org/