From: Michal Suchanek <hramrach@...>
Date: 2007-12-08T19:43:47+09:00
Subject: Re: Default encoding not set properly

On 08/12/2007, Martin Duerst <duerst@it.aoyama.ac.jp> wrote:
> Hello Michal,
>
> First, can you please fix your mail software so that it sends
> standard Internet line endings?

There was no line ending thus it did not send any, sorry.

>
> At 05:21 07/12/08, Michal Suchanek wrote:
> >Hello, I tried running 1.9 irb, and figure out where the default encoding
> >is. > >It looks like it is Encoding.default_external. > >However, it is not set
> >properly when I run irb in luit: > >irb(main):001:0>
> >Encoding.default_external >=> <Encoding:UTF-8> >irb(main):002:0> `locale` >=>
> >"LANG=en_US.UTF-8\nLC_CTYPE=\"cs_CZ\"\nLC_NUMERIC=\"cs_CZ\"\nLC_TIME=\"cs_CZ\"\nLC_COLLATE=\"cs_CZ\"\nLC_MONETARY=\"cs_CZ\"\nLC_MESSAGES=\"cs_CZ\"\nLC_PAPER=\"cs_CZ\"\nLC_NAME=\"cs_CZ\"\nLC_ADDRESS=\"cs_CZ\"\nLC_TELEPHONE=\"cs_CZ\"\nLC_MEASUREMENT=\"cs_CZ\"\nLC_IDENTIFICATION=\"cs_CZ\"\nLC_ALL=cs_CZ\n" > >As
> >I understand it applications should look at LC_CTYPE, LC_ALL, and >LANG (in
> >this order)
>
> Can you point to some source for this information?

http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html#tag_08_02

Actually LC_ALL takes precedence over LC_CTYPE but it is not relevant
for this case.

> Also, the only one of these environment variables that contains an encoding
> is LANG, which says UTF-8, and this is what Ruby took, so I don't see
> what's wrong. Maybe cs_CZ in your case implies another encoding, and
> you are expecting this to be picked up, but how should Ruby actually do this?

The same way as if LANG was cs_CZ.

AFAIK you cannot tell the encoding from the locale name, and the
convention is that locales use some "default encoding for that
language" or specify the encoding in the name. Thus en_US is
ISO8859-1, en_US.ISO8859-15 is the same with the Euro sign (iirc),
en_US.UTF-8 is UTF-8, and cs_CZ is ISO8859-2.

Thanks

Michal