From: Gary Wright <gwtmp01@...>
Date: 2009-06-06T11:20:16+09:00
Subject: Re: rounding errors?


On Jun 5, 2009, at 10:07 PM, Pete Hodgson wrote:

> :~$ ruby -e "p ((0.29)*100).to_i"
> 28
>
> Really?! I know that digital storage of floating point numbers can  
> lead
> to things like this, but this one seems a little extreme to me.


 >> sprintf "%0.30f", 0.29
=> "0.289999999999999980015985556747"
 >> sprintf "%0.30f", 0.29*100
=> "28.999999999999996447286321199499"
 >> sprintf "%0.30f", (0.29*100).to_i
=> "28.000000000000000000000000000000"

If you want to really understand what is going on:
<http://docs.sun.com/source/806-3568/ncg_goldberg.html>

Gary Wright