From: Dan Janowski <danj@3skel.com>
Date: 2004-10-28T03:53:06+09:00
Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show 
the exact value. Why shouldn't to_i benefit?

It seems to me that this type of intolerance would effectively make 
to_i not only worthless but dangerous.

Dan


On Oct 27, 2004, at 14:25, Jamis Buck wrote:

> Dan Janowski wrote:
>> I can understand floating point errors, but this one seems too small 
>> for that.
>> Especially given this:
>> irb(main):023:0> v=100.0*9.95
>> => 995
>> irb(main):024:0> v.to_i
>> => 994
>
> a = 100 * 9.95
> "%.16f" % a
> => "994.9999999999998863"
>
> It's not a problem with Ruby, it's a problem with floating point 
> numbers in general. :(
>
> - Jamis
>
> -- 
> Jamis Buck
> jgb3@email.byu.edu
> http://www.jamisbuck.org/jamis
>