From: Thomas E Enebo <enebo@...>
Date: 2006-03-31T09:13:52+09:00
Subject: Re: Very interesting Float comparison!

On Fri, 31 Mar 2006, Ryan Allen defenestrated me:
> Hi Guys (and Gals!)
> 
> This is my first post on the list, so Hello!
> 
> Something very strange is happening (as far as I can tell) with a  
> float comparison; here is the code in question:
> 
> ("55.59".to_f / 100) == 0.5559    # returns false
> 
> 
> It doesn't seem to like .59's... What did .59 ever do?
> 
> I picked this up while writing a unit test, I've run this code in irb  
> and I'm not sure what's going on...
> 
> Can anyone shed some light on this?

  Looking at some IEEE spec will explain this in agonizing detail, but 
simply stated numbers in a computer are stored in a base 2 format.  The 
math you are doing is base 10.  Not all base 10 values can be perfectly 
represented in base 2.  This can cause calculations to not quite come 
out right.

  A book on numerical analysis can also be a good intro on this stuff.

-Tom

-- 
+ http://www.tc.umn.edu/~enebo +---- mailto:enebo@acm.org ----+
| Thomas E Enebo, Protagonist  | "Luck favors the prepared    |
|                              |  mind." -Louis Pasteur       |