From: Gary Wright <gwtmp01@...>
Date: 2008-12-20T03:08:38+09:00
Subject: Re: about Float


On Dec 19, 2008, at 1:05 AM, Gregory Brown wrote:

> On Tue, Dec 16, 2008 at 8:32 PM, Kyung won Cheon <kdream95@gmerce.co.kr 
> > wrote:
>
>> #########################################################
>> # HELP ME^^
>
> http://docs.sun.com/source/806-3568/ncg_goldberg.html

That is the document I usually send people to also.

I think the critical insight to understand is that floating point  
literals written in decimal notation can not be exactly represented  
internally as a binary value. Since there is no one-to-one  
correspondence between the two representations even the simplest  
expression can introduce rounding or approximation errors.

The real number (1/10) has a finite decimal representation but an  
infinite binary representation so:

   puts 0.1

requires that the string '0.1' be converted to an approximate floating  
point representation of (1/10) and then that representation be  
converted back to decimal number for output.  You get different  
results depending on how much precision you ask for in those  
conversions:

 >> (1..50).each { |p| puts "%0.*f" % [p, 0.1] }
0.1
0.10
0.100
0.1000
0.10000
0.100000
0.1000000
0.10000000
0.100000000
0.1000000000
0.10000000000
0.100000000000
0.1000000000000
0.10000000000000
0.100000000000000
0.1000000000000000
0.10000000000000001
0.100000000000000006
0.1000000000000000056
0.10000000000000000555
0.100000000000000005551
0.1000000000000000055511
0.10000000000000000555112
0.100000000000000005551115
0.1000000000000000055511151
0.10000000000000000555111512
0.100000000000000005551115123
0.1000000000000000055511151231
0.10000000000000000555111512313
0.100000000000000005551115123126
0.1000000000000000055511151231258
0.10000000000000000555111512312578
0.100000000000000005551115123125783
0.1000000000000000055511151231257827
0.10000000000000000555111512312578270
0.100000000000000005551115123125782702
0.1000000000000000055511151231257827021
0.10000000000000000555111512312578270212
0.100000000000000005551115123125782702118
0.1000000000000000055511151231257827021182
0.10000000000000000555111512312578270211816
0.100000000000000005551115123125782702118158
0.1000000000000000055511151231257827021181583
0.10000000000000000555111512312578270211815834
0.100000000000000005551115123125782702118158340
0.1000000000000000055511151231257827021181583405
0.10000000000000000555111512312578270211815834045
0.100000000000000005551115123125782702118158340454
0.1000000000000000055511151231257827021181583404541
0.10000000000000000555111512312578270211815834045410