From: duerst@... Date: 2017-03-18T01:50:33+00:00 Subject: [ruby-core:80215] [Ruby trunk Bug#13321] String#codepoints for one-byte encodings Issue #13321 has been updated by duerst (Martin D��rst). InfraRuby (InfraRuby Vision) wrote: > Please update the documentation for `String#codepoints` too. That says "This is a shorthand for `str.each_codepoint.to_a`". > `String#codepoints` does return (Unicode) codepoints for US-ASCII and ISO-8859-1 as those encodings are the basis of Unicode. Well, yes, and for almost all encodings, the returned values are Unicode code points for the ASCII characters, and for some other encodings, there is a bit more of overlap. I don't think we need to go too much into details. > Maybe add `Encoding#unicode_codepoints?` which returns `true` for these encodings: US-ASCII, ISO-8859-1, UTF-8, UTF-16(BE|LE), UTF-32(BE|LE). There are quite a few other cases where behavior of String methods changes depending on the string's Encoding. I think it would be good to have access to this information, but methods with more general names may be needed. Anyway, to get Unicode codepoints out of an arbitrary string, `string.encode('UTF-8').codepoints` will always do the job. > (Also, there's an unrelated change in that revision.) Yes, thanks for noticing, fixed. ---------------------------------------- Bug #13321: String#codepoints for one-byte encodings https://bugs.ruby-lang.org/issues/13321#change-63650 * Author: InfraRuby (InfraRuby Vision) * Status: Rejected * Priority: Normal * Assignee: * Target version: * ruby -v: * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- On many versions of Ruby, including 2.4.0: "\x80".force_encoding("WINDOWS-1252").codepoints.first # => 0x80 I expected 0x20AC: https://en.wikipedia.org/wiki/Windows-1252 See: https://github.com/ruby/ruby/blob/v2_4_0/string.c#L7817-L7818 https://github.com/ruby/ruby/blob/v2_4_0/string.c#L422-L424 -- https://bugs.ruby-lang.org/ Unsubscribe: