From: Jason DiCioccio <jd@...>
Date: 2004-10-28T05:25:19+09:00
Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994

Thanks for your responses.  I actually did hear about this a long time ago 
I think, but I had forgotten the specifics.  It's really not often that I 
end up dealing with floating point numbers.  Most things are done (for me) 
with Integers it seems.  I'd imagine if I were designing a game engine or 
doing scientific work, that would not be the case :).

Thanks again!
-JD-

--On Thursday, October 28, 2004 05:09:48 +0900 Volkard Henkel 
<volkard@normannia.de> wrote:

> Hello Jason,
>
> Wednesday, October 27, 2004, 9:49:28 PM, you wrote:
>
>> Hello!
>
>> --On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck
>> <jgb3@email.byu.edu> wrote:
>
>>> Bill Atkins wrote:
>>>> OK, that would make sense if v's value was 994.5, but it seems to be
>>>> 995.0.  Why should to_i round down like that?
>>>
>>> But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
>>> as 995.0, rounding it.
>>>
>>> - Jamis
>
>> I'm confused now..  100.0 * 9.95 is clearly 995.  So what exactly is the
>> issue with floating point numbers is making this come out to
>> 994.999999999983 (or wahtever ;))?  If every language is plagued by this
>> problem, then I'd be curious to know the history behind it..
> 9.95 can not exactly put into a floeating point number with base 2.
> with base 10 it's no problem.
> 9.95==9*10^0+9*10^-1+5*10^-2
> but with bas 2 this will become a periaodic dual fraction.
> so 9.95 is the problem. its internally in the computer slightly lower
> than the real 9.95
> multiplying by 100.0 (which can be expresses exactly) does not repair
> the problem of 9.95
>
> every language is plagued by this problem, if it uses the floating
> point arithmetic given by the processor (and doing this is a good idea
> because the fine speed). we as programmers have to know about this
> problem and live with it. mostly its a good idea to ignore it. :)
> sometimes we don't calc 100.0 Dollars * 9.95 Dollars but do 10000
> Pence * 995 Pence to avoid the problem (because irregular
> pence-vanishing with doubles is not amusing, but regular
> pence-vanishing with intagers can be managed).
>
> --
> Best regards,
>  Volkard                            mailto:volkard@normannia.de
>
>