From: Hugh Sasse <hgs@...>
Date: 2006-11-09T02:49:03+09:00
Subject: Re: Simple Math Problem

On Thu, 9 Nov 2006, Tim Pease wrote:

> On 11/8/06, Thom Loring <tloring@mac.com> wrote:
> > Can anyone shed some light on a simple math problem I have encountered?
> > I am doing a very simple calculation, and the comparison of that result
> > compared to the number itself is returning false in some cases...for
> > instance:
> > 
> > $irb
> > >> v1 = 80.04
> > => 80.04
> > 
> > >> v2 = 0.12 * 667
> > => 80.04
> > 
> > >> v1 == v2
> > => false
> > 
> > >> v1 > v2
> > => true
> > 
> > >> v1 < v2
> > => false
> > 
> > >> v1.inspect
> > => "80.04"
> > 
> > >> v2.inspect
> > => "80.04"
> > 
> > Any insight?
> > 
> 
> Machine roundoff error. Floating point values are never precise on any
> computer.
> 
brains hgs 226 %> irb
irb(main):001:0> v1 = 80.04
=> 80.04
irb(main):002:0> v2 = 0.12 * 667
=> 80.04
irb(main):003:0> puts "%.20f" % v1
80.04000000000000625278
=> nil
irb(main):004:0> puts "%.20f" % v2
80.03999999999999204192
=> nil
irb(main):005:0> exit
brains hgs 227 %>

        Hugh