From: "David G. Andersen" <dga@...>
Date: 2004-10-28T03:46:12+09:00
Subject: Re: Rounding error, (100.0 * 9.95).to_i == 994

On Thu, Oct 28, 2004 at 03:25:40AM +0900, Jamis Buck scribed:
> Dan Janowski wrote:
> >I can understand floating point errors, but this one seems too small for 
> >that.
> >Especially given this:
> >
> >irb(main):023:0> v=100.0*9.95
> >=> 995
> >irb(main):024:0> v.to_i
> >=> 994
> 
> a = 100 * 9.95
> "%.16f" % a
> => "994.9999999999998863"
> 
> It's not a problem with Ruby, it's a problem with floating point numbers 
> in general. :(

eep:~> perl
$f = 100.0 * 9.95;
printf("%d\n", $f);
994

if it really bothers you (and you don't want to use round), 
there's an ugly solution:

require 'bigdecimal'
v = BigDecimal.new("9.95")
vone = v * 100
vone.to_i
=> 995

(but really, you just want to use round. ;-)  

  -dave

-- 
work: dga@lcs.mit.edu                          me:  dga@pobox.com
      MIT Laboratory for Computer Science           http://www.angio.net/