From: Michal Suchanek <hramrach@...>
Date: 2006-06-20T02:20:10+09:00
Subject: Re: Unicode roadmap?

On 6/19/06, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:
> Hi,
>
> In message "Re: Unicode roadmap?"
>     on Mon, 19 Jun 2006 21:39:33 +0900, "Michal Suchanek" <hramrach@centrum.cz> writes:
>
> |> a), unless either of strings is "ascii" and the other is "ascii"
> |> compatible.  This point is arguable.
> |
> |What is "ascii"? Specifically I would like string operations to suceed
> |in cases when both strings are encoded as different subset of Unicode
> |(or anything else). ie concatenating an ISO-8859-2 and an ISO-8859-1
> |string sould result in UTF-* string, not an error.
>
> Every encoding has an attribute named ascii_compat.  EUC_JP, SJIS,
> ISO-8859-* and UTF-8 are declared ascii compatible, where EBCDIC,
> UTF-16 and UTF-32 are not.  No other auto conversion shall be done,
> since we don't particularly encourage mixed encoding model.

Reading what you said it appears it would be only possible to add
ascii strings to ascii-compatible sttings. That does not sound very
useful.
If the intended meanig was rather that operations on two
ascii-compatible strings
should always be possible, and that the result is again
ascii-compatible that would sound better.

But it makes these "ascii" encodings a special case. In particular, it
makes UTF-32 less convenient to use.
I guess that for calculation so complex that it would really benefit
form the fast random access of UTF-32 it is reasonable to create a
wrapper that converts the arguments and results. However, If one wants
to perform several such (different) consecutive calculations there are
going to be several useless conversions. It is certainly possible to
make the input interface clever enough to get it right for both UTF-32
and ascii strings but requiring the user to do the conversion on
results does not look nice.

The compatibility could also be just general value that specifies the
encoding family.

ie " ".compatibility => :ascii

ASCII="".encode(:utf8).compatibility

raise "Incompatible encoding #{str.encoding}" unless str.compatibility == ASCII

But different families could be possible. I am not sure if any other
encoding families of any significance exist, though.

Thanks

Michal