From: Hal Fulton <hal9000@...>
Date: 2004-10-28T05:54:30+09:00
Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994

trans. (T. Onoma) wrote:
> On Wednesday 27 October 2004 04:36 pm, Hal Fulton wrote:
> | Jason DiCioccio wrote:
> | > I'm confused now..  100.0 * 9.95 is clearly 995.  So what exactly is the
> | > issue with floating point numbers is making this come out to
> | > 994.999999999983 (or wahtever ;))?  If every language is plagued by this
> | > problem, then I'd be curious to know the history behind it..
> | >
> | :) It's not history, it's math.
> |
> | Here's the short explanation.
> |
> | Remember repeating decimals, which you learned about in elementary school?
> | For example, 1/3 = 0.33333... can't be expressed in a finite number of
> | digits.
> 
> Not exactly, '1/3' is finite. Rational numbers can be expressed. Irrationals 
> cannot. But current standards do not attempt to deal with repeating decimals, 
> even though they could. The reason for this seems to be a matter of history 
> related to an ability to test inequalities quickly.

Expressed as a single number in either base 10 or 2, 0.333... requires an
infinite number of digits (which is all I said).

Knowing that it is rational, you could separately store the two numbers expressing
that ratio. But that is completely different from traditional floating-point storage,
and the simple traditional methods of (e.g.) addition and subtraction would not
work.

> IEEE754 is now 20 years old, and is showing its age. I have actually been 
> thinking of working on a improved version myself. But there's many details to 
> deal with, so who knows... maybe.

Good idea, go for it.

Hal