From: Tim Pease <tim.pease@...>
Date: 2006-11-09T02:59:37+09:00
Subject: Re: Simple Math Problem

On 11/8/06, Hugh Sasse <hgs@dmu.ac.uk> wrote:
> >
> > Machine roundoff error. Floating point values are never precise on any
> > computer.
> >
> brains hgs 226 %> irb
> irb(main):001:0> v1 = 80.04
> => 80.04
> irb(main):002:0> v2 = 0.12 * 667
> => 80.04
> irb(main):003:0> puts "%.20f" % v1
> 80.04000000000000625278
> => nil
> irb(main):004:0> puts "%.20f" % v2
> 80.03999999999999204192
> => nil
> irb(main):005:0> exit
> brains hgs 227 %>
>

With IEEE 754  floating point values you only have 15 digits of
precision (8 byte doubles). Ara's little example above demonstrates
that very nicely. Things go pear shaped in the 17th digit with the
"exact" value 80.04 (v1).

Take a look at the "assert_in_delta" method of the Test::Unit library.
That performs floating point comparison by comparing a diff against
some epsilon (delta).

<shameless plug>
There is an RCR open to implement a =~ operator for floating point
values that would perform a diff comparison against an epsilon value.

http://rcrchive.net/rcr/show/340
</shameless plug>

Blessings,
TwP