From: Yossef Mendelssohn <ymendel@...>
Date: 2007-10-23T05:27:54+09:00
Subject: computers + floating-point = a chore

I recall seeing a few posts about the "fun" involving floating-point
representation of numbers, but this is something I found a little
surprising:

Cassady:~ yossef$ irb --prompt simple
>> require 'bigdecimal'
=> true
>> bd = BigDecimal('19.76')
=> #<BigDecimal:9ccc,'0.1976E2',8(8)>
>> bd <=> 19.76
=> -1
>> bd.to_f <=> 19.76
=> -1
>> bd.to_f
=> 19.76
>> 19.76 <=> 19.76
=> 0

Odd, isn't it? What's going on here?

>> 19.75999999999999999999999999999999999
=> 19.76

Ah, maybe this is it.

>> 19.75999999999999999999999999999999999 <=> 19.76
=> 0

Nope!
Well, a bit of experimentation led me to this, which demonstrates two
threshold points.

>> 19.7599999999999
=> 19.7599999999999
>> 19.75999999999999
=> 19.76
>> 19.75999999999999 <=> 19.76
=> -1
>> 19.759999999999999 <=> 19.76
=> -1
>> 19.7599999999999999 <=> 19.76
=> 0

It appears 19.75999999999999 (that's 12 9s, or 14 decimal places
total) is just close enough to *display* rounded up, but not actually
*become* that number. Adding two more 9s makes it close enough to
actually become that number.

No question or call for help, really. I just wanted to point out
something that intrigued me.

--
-yossef