From: Chris Pearl <chrispearl@...>
Date: 2006-12-01T10:16:36+09:00
Subject: Re: floating point problem... floating indeed :(

Someone else might wish to give a more concrete explanation, but as a
rule, floating-point calculations - using the usual Float datatype -
are not accurate, and prone to rounding errors such as the above.

If you're dealing with limited precision (i.e. X digits after the
decimal point) you should consider using the Decimal datatype instead.

On 12/1/06, teeshift <teeshift@gmail.com> wrote:
> I recently wrote a code and haven't been found the bug and start
> suspecting that it was
> the floating computation that causing the problem. I did a little
> experiement and found the
> following phenomena:
>
> I am using
>
> Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
> GNU/Linux
> Ubuntu is the OS.
>
> #####################
> #!/usr/bin/ruby1.9
>
> a = 1.0
>
> a.to_f
>
> while(a < 15.0)
>   a = a + 0.1
>   print a,"\t",a * a,"\n"
> end
> #####################
>
> and the output is...(as following) quite odd.  Anyone can explain it?
> or a way to avoid it (while
> keeping the precision)
>
>
> 1.1     1.21
> 1.2     1.44
> 1.3     1.69
> 1.4     1.96
> 1.5     2.25
> 1.6     2.56
> 1.7     2.89
> 1.8     3.24
> 1.9     3.61
> 2.0     4.0
> 2.1     4.41
> 2.2     4.84
> 2.3     5.29000000000001
> 2.4     5.76000000000001
> 2.5     6.25000000000001
> 2.6     6.76000000000001
> 2.7     7.29000000000001
> 2.8     7.84000000000001
> 2.9     8.41000000000001
> 3.0     9.00000000000001
> 3.1     9.61000000000001
> 3.2     10.24
> 3.3     10.89
> 3.4     11.56
> 3.5     12.25
> 3.6     12.96
> 3.7     13.69
> 3.8     14.44
> 3.9     15.21
> 4.0     16.0
> 4.1     16.81
> 4.2     17.64
> 4.3     18.49
> 4.4     19.36
> 4.5     20.25
> 4.6     21.16
> 4.7     22.09
> 4.8     23.04
> 4.9     24.01
> 5.0     25.0
> 5.1     26.01
> 5.2     27.04
> 5.3     28.09
> 5.4     29.16
> 5.5     30.25
> 5.6     31.36
> 5.7     32.49
> 5.8     33.64
> 5.9     34.81
> 6.0     35.9999999999999
> 6.1     37.2099999999999
> 6.19999999999999        38.4399999999999
> 6.29999999999999        39.6899999999999
> 6.39999999999999        40.9599999999999
> 6.49999999999999        42.2499999999999
> 6.59999999999999        43.5599999999999
>
>
>