From: "David G. Andersen" <dga@...>
Date: 2004-10-28T04:59:10+09:00
Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994

On Thu, Oct 28, 2004 at 04:49:28AM +0900, Jason DiCioccio scribed:
> 
> I'm confused now..  100.0 * 9.95 is clearly 995.  So what exactly is the 
> issue with floating point numbers is making this come out to 
> 994.999999999983 (or wahtever ;))?  If every language is plagued by this 
> problem, then I'd be curious to know the history behind it..

You can't represent all floating point numbers accurately in
binary.  The problem in this case isn't actually in the storage
of 9.95 or 995 in floating point;  it comes in when you do the
multiplication.  You might take a peek at

http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

It has a nice overview of some of the pitfalls of floating point,
and the links go on from there...

  -Dave

-- 
work: dga@lcs.mit.edu                          me:  dga@pobox.com
      MIT Laboratory for Computer Science           http://www.angio.net/