From: Martin Duerst <duerst@...>
Date: 2008-09-19T17:11:26+09:00
Subject: [ruby-core:18725] Re: Character encodings - a radical  suggestion

At 10:21 08/09/18, Urabe Shyouhei wrote:

>Have you read Matz's post abount yen sign problem?

The yen sign problem is indeed a big problem. It's similar to the
Y2K problem (people knew they shouldn't use just two digits, but they
did, and people know they shouldn't use 0x5c for the Japanese
currency anymore, but they still do), except that there is no deadline,
and so there is not enough pressure to fix it.

>Converter IS a
>problem; you cannot make a converter over (Encoding A -> Unicode ->
>Encoding A).

Sorry, but for all the encodings in daily use, including those in
Japan, round-tripping via Unicode works fine. Unicode was explicitly
designed to do that (at the expense of introducing quite a bit of
what some people might call garbage). This very much includes the
Yen/Backslash. The problems may start when you try to do some processing.
(many kinds of processing are not affected, but some are)

>That must lose some input.  Data loss is the worst thing
>to introduce, so ruby asks you to take the risk by explicitly calling a
>conversion method.

Taking the risk explicitly is fine. But some people may feel that it's
easier to do that application-by-application than string by string.


>Problems on character encodings are sourced from complexities of human
>activities.

Very much so indeed.

>I can hardly believe there are any simple, perfect, and/or
>"neat" solution.

Who said Unicode is neat? It's just that sometimes one messy
solution is better than a mess of many solutions :-(.

Regards,   Martin.



#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp      mailto:duerst@it.aoyama.ac.jp