From: Yukihiro Matsumoto <matz@...>
Date: 2008-09-19T23:32:50+09:00
Subject: [ruby-core:18744] Re: Character encodings - a radical  suggestion

Hi,

In message "Re: [ruby-core:18742] Re: Character encodings - a radical  suggestion"
    on Fri, 19 Sep 2008 22:40:20 +0900, Dave Thomas <dave@pragprog.com> writes:

|I'm no expert in any of this, but I wonder if part of the problem  
|might be that Ruby tries to support all encodings both internally and  
|externally. Might it be easier to support the full set externally, but  
|to have a more limited set internally?

That what we thought.  Limited support for UTF-16 and dummy encodings
is our result, which seems to be imperfect.

|For example, you could support  
|UTF-16<any endian> as an external encoding, but transcode to UTF-8 on  
|the way in. You could still support a rich variety of internal  
|encodings, including the Asian ones you need. But you wouldn't have to  
|deal with UTF-16 when implementing Regexp#escape :)  So, keep the  
|current set of encodings, but only allow a reasonable (ASCII- 
|compliant) subset as internal encodings.

I think you've suggested something valuable, but I cannot imagine the
detail (yet).  Magically transcode text in unsupported encoding to
certain supported encoding.  Hmm.

UTF-16 and UTF-8 are easier set, since they are semantically same.
But how should we treat ISO-2022-JP for example?
# No, I am asking myself, not you, Dave.

							matz.