From: matz@... (Yukihiro Matsumoto)
Date: 2002-11-29T17:56:08+09:00
Subject: Re: UTF-8 "bug": not in accordance with the unicode-3 specs

Hi,

In message "UTF-8 "bug": not in accordance with the unicode-3 specs"
    on 02/11/29, "Paul Melis" <paul@floorball.nl> writes:

|Seems to me UTF-8 is just a way of encoding multi-byte characters with a
|stream of bytes, but it looks like the Unicode standard has some further
|restrictions on what is a legal stream and what is not.

Array#pack("U") and String#unpack("U") are UTF-8 packer/unpacker, so
that it should not restricted by Unicode standard additional
restriction.   But I will add checks for malformed UTF-8 and redundant
UTF-8 sequences (warnings with -w option).

I think checks for surrogate pair values (from U+D800 to U+DFFF)
should be done in different layer, which handles Unicode semantics.

							matz.