From: Tanaka Akira <akr@...>
Date: 2013-05-05T07:56:05+09:00
Subject: [ruby-core:54786] Re: Call setlocale(LC_ALL, "") instead of setlocale(LC_CTYPE, "")

2013/5/5 Nikolai Weibull <now@disu.se>:
>
> Why isn�t setlocale called with LC_ALL instead of LC_CTYPE in main.c?
>
> I have a faint recollection that I�ve discussed this earlier, but I
> couldn�t find it in the archives.  I did find the following bug
> report, though:
>
>> Issue #1528 has been updated by Shyouhei Urabe.
>
>> Without any setlocale call programs should explicitly behave as in "C" locale.  No matter of luck are involved in it.
>
>> Historically Ruby has suffered bugs and pitfalls around locales.  I do not want them to raise from the dead, and sticking onto "C" locale is a secure way to prevent that.
>
> I hope that this, four years later, isn�t true anymore and that you�ll
> change LC_CTYPE to LC_ALL.
>
> If motivation really is necessary, then, for example, strcoll won�t
> produce results in the user�s chosen locale if the call to setlocale
> doesn�t include LC_COLLATE.

Changing setlocale(LC_CTYPE, "") to setlocale(LC_ALL, "") still cause problems.

make check with changed Ruby on GNU/Linux fails as follows with
tr_TR.UTF-8 locale.
(tr_TR locale changes case related bahavior.  [ruby-core:14662])

% LANG=tr_TR.UTF-8 make check
...
# Running tests:

[  247/13312] DL::TestDL#test_call_double = 0.09 s
  1) Failure:
DL::TestDL#test_call_double [/home/akr/ruby/tst1/ruby/test/dl/test_dl2.rb:90]:
Expected |0.1 - 0.0| (0.1) to be <= 0.001.

[  279/13312] DL::TestFunc#test_strtod = 0.09 s
  2) Failure:
DL::TestFunc#test_strtod [/home/akr/ruby/tst1/ruby/test/dl/test_func.rb:152]:
Expected |12.34 - 12.0| (0.33999999999999986) to be <= 0.001.

[  439/13312] Fiddle::TestFunc#test_strtod = 0.09 s
  3) Failure:
Fiddle::TestFunc#test_strtod
[/home/akr/ruby/tst1/ruby/test/fiddle/test_func.rb:56]:
Expected |12.34 - 12.0| (0.33999999999999986) to be <= 0.001.

[ 2133/13312] TestArgf#test_unreadable = 0.00 s
  4) Error:
TestArgf#test_unreadable:
ArgumentError: invalid byte sequence in US-ASCII
    /home/akr/ruby/tst1/ruby/test/ruby/test_argf.rb:803:in `block in
test_unreadable'
    /home/akr/ruby/tst1/ruby/test/ruby/test_argf.rb:801:in `each'
    /home/akr/ruby/tst1/ruby/test/ruby/test_argf.rb:801:in `test_unreadable'
    /home/akr/ruby/tst1/ruby/lib/test/unit/testcase.rb:17:in `run'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:657:in `block in _run_suites'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:655:in `each'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:655:in `_run_suites'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:21:in `run'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:774:in `run'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:834:in `run'
    /home/akr/ruby/tst1/ruby/lib/test/unit.rb:838:in `run'
    ./test/runner.rb:17:in `<main>'

[ 2379/13312] TestBigDecimal#test_BigMath_exp = 0.00 s
  5) Failure:
TestBigDecimal#test_BigMath_exp
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1280]:
Expected |485165195.4097903 - 0.48516519540979027797E9|
(485165195.4097903) to be <= 0.48516519540979027797E6.

[ 2383/13312] TestBigDecimal#test_BigMath_log_with_2 = 0.00 s
  6) Failure:
TestBigDecimal#test_BigMath_log_with_2
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1365]:
Expected |0.6931471805599453 -
0.6931471805599453094172321214581765674099931010273832E0|
(0.6931471805599453) to be <= 0.001.

[ 2384/13312] TestBigDecimal#test_BigMath_log_with_42 = 0.01 s
  7) Failure:
TestBigDecimal#test_BigMath_log_with_42
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1375]:
Expected |3.7376696182833684 -
0.3737669618283368305917830101823881909945797027394132E1|
(3.7376696182833684) to be <= 0.001.

[ 2386/13312] TestBigDecimal#test_BigMath_log_with_exp_1 = 0.00 s
  8) Failure:
TestBigDecimal#test_BigMath_log_with_exp_1
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1361]:
Expected |1.0 -
0.10000000000000000000146094228462874163031819506060204E1| (1.0) to be
<= 0.001.

[ 2393/13312] TestBigDecimal#test_BigMath_log_with_reciprocal_of_42 =
0.00 s
  9) Failure:
TestBigDecimal#test_BigMath_log_with_reciprocal_of_42
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1381]:
Expected |-96.70857390574992 -
-0.96708573905749918728755641096743294420914068228418976E2|
(96.70857390574992) to be <= 0.001.

[ 2425/13312] TestBigDecimal#test_exp_with_1 = 0.00 s
 10) Failure:
TestBigDecimal#test_exp_with_1
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1275]:
Expected |2.718281828459045 - 0.27182818284590452354E1|
(2.718281828459045) to be <= 0.002718281828459045.

[ 2469/13312] TestBigDecimal#test_power_of_three = 0.00 s
 11) Failure:
TestBigDecimal#test_power_of_three
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1007]:
Expected |0.012345679012345678 -
0.12345679012345679012345679012345679012345679012345679012345679E-1|
(0.012345679012345678) to be <= 0.001.

[ 2488/13312] TestBigDecimal#test_sqrt_bigdecimal = 0.00 s
 12) Failure:
TestBigDecimal#test_sqrt_bigdecimal
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:786]:
Expected |0.3 - 0.3E0| (0.3) to be <= 0.001.

[ 2495/13312] TestBigDecimal#test_to_f = 0.00 s
 13) Failure:
TestBigDecimal#test_to_f
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:557]:
<1.0> expected but was
<0.0>.

[ 2510/13312] TestBigMath#test_atan = 0.00 s
 14) Failure:
TestBigMath#test_atan
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:57]:
Expected |0.7853981633974483 -
0.78539816339744830961566084581987572097996482338035675E0|
(0.7853981633974483) to be <= 0.001.

[ 2511/13312] TestBigMath#test_const = 0.00 s
 15) Failure:
TestBigMath#test_const
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:13]:
Expected |3.141592653589793 -
0.3141592653589793238462643383279502883919859293521427E1|
(3.141592653589793) to be <= 0.001.

[ 2512/13312] TestBigMath#test_cos = 0.00 s
 16) Failure:
TestBigMath#test_cos
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:42]:
Expected |1.0 - 0.1E1| (1.0) to be <= 0.001.

[ 2513/13312] TestBigMath#test_sin = 0.00 s
 17) Failure:
TestBigMath#test_sin
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:29]:
Expected |0.7071067811865476 -
0.70710678118654752440084436210484903917782034136995079E0|
(0.7071067811865476) to be <= 0.001.

[ 2514/13312] TestBigMath#test_sqrt = 0.00 s
 18) Failure:
TestBigMath#test_sqrt
[/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:18]:
Expected |1.4142135623730951 -
0.141421356237309504880168872420969807825E1| (1.4142135623730951) to
be <= 0.001.

[ 4903/13312] TestGemExtCmakeBuilder#test_self_build = 0.02 s
 19) Skipped:
TestGemExtCmakeBuilder#test_self_build
[/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]:
cmake not present

[ 4904/13312] TestGemExtCmakeBuilder#test_self_build_fail = 0.02 s
 20) Skipped:
TestGemExtCmakeBuilder#test_self_build_fail
[/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]:
cmake not present

[ 4905/13312] TestGemExtCmakeBuilder#test_self_build_has_makefile = 0.03 s
 21) Skipped:
TestGemExtCmakeBuilder#test_self_build_has_makefile
[/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]:
cmake not present

[ 4994/13312] TestGemInstaller#test_build_extensions_extconf_bad =
0.10 s
 22) Failure:
TestGemInstaller#test_build_extensions_extconf_bad
[/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_installer.rb:86]:
Expected /\/home\/akr\/ruby\/tst1\/ruby\/ruby: No such file/ to match
"/home/akr/ruby/tst1/ruby/ruby
extconf.rb\n/home/akr/ruby/tst1/ruby/ruby: B�yle bir dosya ya da dizin
yok -- extconf.rb (LoadError)\n".

[ 7181/13312] TestMiniTestUnitTestCase#test_capture_subprocess_io =
0.00 s
 23) Skipped:
TestMiniTestUnitTestCase#test_capture_subprocess_io
[/home/akr/ruby/tst1/ruby/test/minitest/test_minitest_unit.rb:1401]:
Dunno why but the parallel run of this fails

[ 8173/13312] TestProcess#test_execopts_env = 0.07 s
 24) Error:
TestProcess#test_execopts_env:
ArgumentError: invalid byte sequence in US-ASCII
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:260:in `sub'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:260:in `block
in test_execopts_env'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:33:in `block (2
levels) in with_tmpchdir'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:32:in `chdir'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:32:in `block in
with_tmpchdir'
    /home/akr/ruby/tst1/ruby/lib/tmpdir.rb:88:in `mktmpdir'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:30:in `with_tmpchdir'
    /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:251:in
`test_execopts_env'

[10528/13312] TestRakeFunctional#test_signal_propagation_in_testsTerminated
 = 0.08 s
 25) Skipped:
TestRakeFunctional#test_signal_propagation_in_tests
[/home/akr/ruby/tst1/ruby/test/rake/test_rake_functional.rb:436]:
Signal detect seems broken on this system

Finished tests in 557.567111s, 23.8752 tests/s, 4613.0285
assertions/s.
13312 tests, 2572073 assertions, 18 failures, 2 errors, 33 skips

ruby -v: ruby 2.1.0dev (2013-05-04 trunk 40574) [x86_64-linux]
gmake: *** [yes-test-all] Hata 20
% ./ruby -v
ruby 2.1.0dev (2013-05-04 trunk 40574) [x86_64-linux]

It seems four years are not enough to make ruby robust with locales.

I think it is acceptable that make ruby more robust.
(i.e. fixing above failures and errors are ok.)

But changing setlocale invocation to LC_ALL is too dangerous.
The change affects not only ruby itself but also other extension libraries.
-- 
Tanaka Akira