From: ara.t.howard@...
Date: 2006-03-10T02:06:08+09:00
Subject: Re: ruby float calculations

On Fri, 10 Mar 2006, Alexandru Toma wrote:

> Could someone please explain to me why the following happens:
>
> irb(main):001:0> 0.29 - 0.38 + 0.1
> => 0.00999999999999998
> irb(main):002:0> 0.29 + (-0.38 + 0.1)
> => 0.00999999999999995
> irb(main):003:0> 0.29 + 0.1 - 0.38
> => 0.01
>
> How can I prevent something like this from happening, or at the very
> least how can I work around such errors?
>
> Many thanks in advance.

it is a hardware limitation, even a C program will do the same:

     harp:~ > cat a.c
     #include <stdlib.h>
     #include <stdio.h>

     main () {
       printf ("%32.32f\n", 0.29 - 0.38 + 0.1);
       printf ("%32.32f\n", 0.29 + (-0.38 + 0.1));
       printf ("%32.32f\n", 0.29 + 0.1 - 0.38);
     }


     harp:~ > gcc a.c && a.out
     0.00999999999999998112620858137234
     0.00999999999999995337063296574343
     0.01000000000000000888178419700125


with ruby, at least, you can use BigDecimal or the like - but with a serious
speed penalty.  in short this is just the way computers work:

   http://en.wikipedia.org/wiki/Floating_point

hth.

-a

-- 
knowledge is important, but the much more important is the use toward which it
is put.  this depends on the heart and mind of the one who uses it.
- h.h. the 14th dali lama