From: Volkard Henkel Date: 2004-10-28T05:09:48+09:00 Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994 Hello Jason, Wednesday, October 27, 2004, 9:49:28 PM, you wrote: > Hello! > --On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck > wrote: >> Bill Atkins wrote: >>> OK, that would make sense if v's value was 994.5, but it seems to be >>> 995.0. Why should to_i round down like that? >> >> But it isn't 995.0. It is 994.99999999883 or something. IRB displays it >> as 995.0, rounding it. >> >> - Jamis > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the > issue with floating point numbers is making this come out to > 994.999999999983 (or wahtever ;))? If every language is plagued by this > problem, then I'd be curious to know the history behind it.. 9.95 can not exactly put into a floeating point number with base 2. with base 10 it's no problem. 9.95==9*10^0+9*10^-1+5*10^-2 but with bas 2 this will become a periaodic dual fraction. so 9.95 is the problem. its internally in the computer slightly lower than the real 9.95 multiplying by 100.0 (which can be expresses exactly) does not repair the problem of 9.95 every language is plagued by this problem, if it uses the floating point arithmetic given by the processor (and doing this is a good idea because the fine speed). we as programmers have to know about this problem and live with it. mostly its a good idea to ignore it. :) sometimes we don't calc 100.0 Dollars * 9.95 Dollars but do 10000 Pence * 995 Pence to avoid the problem (because irregular pence-vanishing with doubles is not amusing, but regular pence-vanishing with intagers can be managed). -- Best regards, Volkard mailto:volkard@normannia.de