From: Michal Suchanek <hramrach@...>
Date: 2008-01-18T21:32:23+09:00
Subject: Re: does ruby's strftime not attempt POSIX-compliance?

On 17/01/2008, Jochen Hayek <jochen+ruby-forum@hayek.name> wrote:
> Michal Suchanek wrote:
>
> > On 15/01/2008, Jochen Hayek wrote:
>
> > There might be pieces of text or other data in different languages.
> > And POSIX locale handles such situations poorly.
> > And you should remember that the application
> > is not just the interface.
>
> Right, but human read oriented text strings
> will only occur in "*the* (user) interface",
> not in the number crunching part of the software, I assume,
> just keeping to "good practices" ...

There are also string crunching applications.

>
> > It's the mistake that the people designing POSIX locale did:
> > they forgot that the data has to live somewhere
> > before it gets to the user interface.
>
> Right, and the user interface is the right place,
> to convert the internal time value into a locale oriented string.
> Just as in MVC: the View is the right place to deal with that stuff.

yes, the problem is that you have to deal with strings also outside of
view, and you want to do that independently of the view and its
locales.

>
> That's what I like and appriate with academical people:
> they are not pragmatic, and they don't need to be ;-)
> And it's good that way. Serious.
>
> Dear Michal, you are right in that it should be dealt with software with
> multiple user interfaces,
> all dealing with users in entirely different locales.

No, I meant software dealing with texts in different languages.
Different languages have different sorting rules (even for the same
letters) but you have only one locale. That's why I don't really like
the idea of using locale too much.

>
> BUT: back in real life ... different instances of a single program can
> very well deal each with a different locale -- w/o conflict and
> overlaps.
>
> >> > I suspect you misunderstand the way locale is currently handled.
> >> > It is not used by ruby, it is avoided.
> >>
> >> Well, as long as MRI is written in C
> >> and makes use of host libraries esp. like (g)libc,
> >> "avoiding" is better replaced by "defaulting",
> >
> > no, it's avoiding it. The locale was set to "C" in 1.8,
> > and now only LC_CTYPE is set.
>
> Oh, and that's not set *to* *a* *value*?
>
> Matz wrote, that this is done:
>
>   setlocale(LC_CTYPE, "");

Yes, it's correct. It means "set locale from the environment variable"

>
> >> because *there* *are* language strings emitted by C library routines
> >> and passed through to ruby class resp. object methods,
> >> and it's probably not too wrong
> >> to assume these strings are "very similar" to en_US.
> >> That is a locale de facto, isn't it?!?
> >
> > no, locale is about having the language specific stuff different at
> > different times. Before there was locale, everything was in the
> > implicit "C" locale which was like en_US
>
> As mentioned above:
> For me as a pragmatic programmer :-)
> it's quite sufficient, that an instance of a program
> initially acquires a single locality and keeps that for its life-time
> and that's it.
> No changes in the meantime.

With ruby 1.9 you get exactly that.

>
> >> > The only recent change was adding the
> >> > setlocale() call in the ruby interpreter
> >> > that is required for some
> >> > extensions to work properly in different locales.
> >>
> >> Extensions built on assumptions *like* "you always work with en_US",
> >> of course, that simplifies life tremendously,
> >> but you don't real want to discuss this bad idea with me, right?
> >
> > No, extensions that use libraries built around the assumption that
> > LC_CTYPE specifies the correct character classes which the "C" locale
> > does not for most cases.
>
> After googling a while for ruby, rails, and locale,
> my impression was very much different,
> but I am running out of time right now
> and you can do the query yourself,
> and I prefer to give in here.
>
> Just a single and last example:
>
>   $ env LC_ALL=fr_FR /usr/local/ruby1.9/bin/ruby \
>     -e 't = Time.now; puts t.strftime("%A")'
>   Thursday

Yes, this is ruby's strftime that operates on ruby Time values and has
nothing in common with the libc strftime except the name ;-)

>
>   $ env LC_ALL=fr_FR date '+%A'
>   jeudi

That's fine. However, if you use strftime for some text based protocol
or file format you are in trouble here. And you have only one locale
so you cannot choose when you get the French representation, and when
the English one.

That's why I suggest extending ruby-locale to provide Locale::strftime
or Time#locale_strftime that converts the Time value to the C time
value and performs C strftime on it with the locale and platform
specific result. I am not even sure that strftime is localized on all
platforms to which ruby is ported.

>
> The second thing is exactly what a simple UNIX programmer expects,
> the first one is r***ish.
>
> So, now that's not twisting with environment variables, right? :-)

Yes, they are not twisted in any way.

>
> I am sorry,
> but this should get fixed,
> and then we can proceed discussing the matter.
>
> >> * passing (g)libc capabilities straight through,
> >> where not too much speaks against it
> >
> > The libc is never used directly,
> > because it does not operate on the same data types as ruby does.
> > You could make wrappers but it's always some *addition*.
>
> And that implies it's better to reinvent the wheel
> instead of using available basic middleware?

No I am not in favour of reinventing the wheel here.

>
> >> * implement nicer locale capabilities,
> >> when there are any coding resources left
> >
> > What capabilities do you need exactly?
>
> *I* would be entirely satisfied,
> if (g)libc's locale capabilities would get passed through
> unchanged, unfiltered, untwisted, un-... (whatsover).
> I think, I have made my point clear by now.

You cannot just pass them. It's not too hard to convert limited subset
of possible ruby valus to C values and use the C localized functions
on them. But you probably cannot format all Time values, and certainly
not all Bignums.

>
> *You* seem unhappy with my simple POSIX approach
> and you request "multi-threaded" capabilities.
>
> Let's not get confused here!
>
> >> You do see, how people struggle for I18N in rails apps in a desperate
> >> and strange way,
>
> Pls go to http://en.wikipedia.org/wiki/I18n
> and read up, what's implied with I18N.
> Not just mulitbyte encodings, but also date/time formats, ...
> That's what I keep referring to.

I am not very fond of localized date/time formats. I find them
confusing. Anybody and anything should be able to parse 2008-01-18,
and it even sorts correctly in dictionary order. Still it should not
be too hard to use the C strftime if anybody wants to do so.

You are welcome to update the ruby-locale extension if you are so concerned :)

[...]

Thanks

Michal