From: Bill Kelly <billk@...>
Date: 2005-06-09T10:57:58+09:00
Subject: Re: Newb math question

Hi Brian,

From: "ES" <ruby-ml@magical-cat.org>
>
> Le 9/6/2005, "brian" <brian@le-roy.org> a �crit:
> >Can someone tell me why this code:
> >    puts (9.0-8.9).to_s
> >results in:
> >    0.0999999999999996
> >
> >I'm expecting 0.01 as the result.  Do I have to specify the precision?
> >Even with the full precision available on the platform (Windows XP), the
> >correct answer is still 0.01, not what's printed.  What's the deal?  I
> >like this language so far, but this is confusing...
>
> Floating point math :/ You can use BigDecimal to avoid most problems.

If you'd like more info about the problem, the first
paragraphs of the section entitled Rounding Error in
this document give the gist of it:
http://docs.sun.com/source/806-3568/ncg_goldberg.html



Regards,

Bill