From: "NARUSE, Yui" <naruse@...>
Date: 2008-06-19T01:44:19+09:00
Subject: [ruby-core:17326] Re: [Ruby 1.9 - Bug #178] (Closed) File.open  on sprintf-formatted string fails with encoding conversion error on OS  X

Martin Duerst wrote:
>>> It may be that it is an attempt to refer to the fact that UTF-8
>>> usually is used in decomposed form (NFD) on the Mac. But that
>>> would not be relevant for opening a file, because the Mac OS
>>> accepts any kind of normalization, and converts to NFD by itself
>>> (similar to a file system that accepts both upper- and lower-case,
>>> but internally uses only one case).
>> Yeah, that's true when you write to filesystem,
>> but when you read from filesystem you may want to know
>> whether they are composed or decomposed.
> 
> That may indeed be the case. But this really only applies to
> filenames (and maybe similar names of resources) on the Mac.
> For such a small subset of data, I think it's overkill if
> as a consequence, processing together with other data is
> blocked (as we saw in the bug report).

This bug is derived from other point.

> As far as I understand, it doesn't apply to file contents or
> other data on the Mac. Also, as soon as you concatenate two
> strings, there is no guarantee that NFD is kept (unless of
> course you implement separate string concatenation for this
> specific encoding). In general, the best thing to do if you
> want to know is to check, and the best thing if you want to
> be sure is to check, and then to change if necessary. But we
> still have to implement this (maybe for -3?).

Off cource, the encoding of other data on the mac may be
other than UTF8-MAC: that's may be composed UTF-8.
I intend that strings labeld as UTF8-MAC may needed to be
converted or normalized.  If you don't care about it,
you can use force_encoding.

> [Also, if the meaning of UTF8-MAC is really NFD, it might
> be better to actually call it that way.]

not real NFD, Apple's NFD as I commented in enc/utf_8.c.

-- 
NARUSE, Yui  <naruse@airemix.jp>