From: Michal Suchanek <hramrach@...>
Date: 2006-06-15T20:21:20+09:00
Subject: Re: Unicode roadmap?

On 6/15/06, Julian 'Julik' Tarkhanov <listbox@julik.nl> wrote:
>

> 5. Preferably separate (and strictly purposed) Bytestring that you
> get out of Sockets and use in Servers etc. - or the ability to
> "force" all strings recieved from external resources to be flagged
> uniformly as being of a certain encoding in _your_ program, not
> somewhere in someone's library. If flags have to be set by libraries,
> they won't be set because most developers sadly don't care:
>
> http://www.zackvision.com/weblog/2005/11/mt-unicode-mysql.html
> http://thraxil.org/users/anders/posts/2005/11/01/unicodification/

Where else should the strings be flagged? If you get a web page
through http request, and the library parses the response for you, it
should set enconding on the web page. You would never know since you
only received the page, not the header.

>
> 6. Unicode-aware strip dealing with weirdo whitespaces (hair space,
> thin space etc.)
> 7. And no, as I mentioned - it doesn't handle it properly because
> the /i modifier is broken, and to deal without it you need to
> downcase BOTH the regexp and the string itself. Closed circle - you
> go and get the Unicode gem with tables.
>
> All of this can be controlled either per String (then 99 out of 100
> libraries I use will be getting it wrong - see above) or by a global
> setting such as $KCODE.

I do not see why libraries should be always wrong. After all, you can
always fix them. And setting the encoding globally is a bad thing. You
cannot have strings encoded in different encodings in one process
then. It looks quite limiting. For one, the web pages that you get
from various servers (and even the same server) can be in varoius
encodings.

Thanks

Michal