From: Tim Pease <tim.pease@...>
Date: 2006-11-09T02:43:03+09:00
Subject: Re: Simple Math Problem

On 11/8/06, Thom Loring <tloring@mac.com> wrote:
> Can anyone shed some light on a simple math problem I have encountered?
> I am doing a very simple calculation, and the comparison of that result
> compared to the number itself is returning false in some cases...for
> instance:
>
> $irb
> >> v1 = 80.04
> => 80.04
>
> >> v2 = 0.12 * 667
> => 80.04
>
> >> v1 == v2
> => false
>
> >> v1 > v2
> => true
>
> >> v1 < v2
> => false
>
> >> v1.inspect
> => "80.04"
>
> >> v2.inspect
> => "80.04"
>
> Any insight?
>

Machine roundoff error. Floating point values are never precise on any computer.

The best way to compare two floating point values is to take there
difference and make sure it lies withing some acceptable error limit.

v1 = 80.04
v2 = 0.12 * 667
diff = v1-v2
diff.abs < Float::EPSILON

That last line returns false, so you need to choose a different epsilon ...

diff.abs < 1.0e-12

That will return true.

Blessings,
TwP