From: John Joyce <dangerwillrobinsondanger@...>
Date: 2007-09-30T04:13:17+09:00
Subject: Re: Unicode

>
> Yes but what about stuff already encoded in UTF-16?

That's why I said read up on unicode!
After you read that stuff you'll understand why it's no problem.
I'm not going to explain it. Many people understand it, but when  
explaining it might make mistakes.
Read the unicode stuff carefully. It's vital for many things.

The only thing you might run into is BOM or Endian-ness, but it's  
doubtful it will be an issue in most cases.

This might get you started.
http://www.unicode.org/faq/utf_bom.html#37


Even Joel Spoelsky wrote a brief bit on unicode... mostly trumpeting  
how programmers need to know it and how few actually do.
The short version is that UTF-16 is basically wasteful. It uses 2  
bytes for lower-level code-points (the stuff also known as ASCII  
range) where UTF-8 does not.

You really need to spend an afternoon reading about unicode. It  
should be required in any computer science program as part of an  
encoding course, Americans in particular are often the ones who know  
the least about it....