From: Markus <markus@...>
Date: 2004-10-16T03:08:48+09:00
Subject: Re: Rounding to X digits

On Friday 15 October 2004 11:14 am, Ara.T.Howard@noaa.gov wrote:
|    jib:~/eg/ruby > ruby truncate.rb
|    134.4
|    134.4

On Fri, 2004-10-15 at 10:27, trans. (T. Onoma) wrote:
> Sorry, I don't follow. Isn't that answer incorrect? The mid-point should 
> round up, not down. At least, that's what I was taught in school.

     But this is truncate, not round.

On Fri, 2004-10-15 at 10:27, trans. (T. Onoma) wrote:
> On Friday 15 October 2004 12:02 pm, Markus wrote:
> |      Peeling one layer of the onion, they differ because:
> |
> | irb(main):033:0> printf "%20.15f",(134.45 * (1.0/0.1))
> | 1344.500000000000000=> nil
> | irb(main):034:0> printf "%20.15f",(134.45/0.1)
> | 1344.499999999999773=> nil
> 
> That isn't too... er... reassuring.

     Yeah.

> | I suppose this is not unexpected (my mama warned me 'bout floats) but it
> | is a little unexpected--no, I'm wrong, 1/5 is a repeating decimal base
> | 2, so it's perfectly expected. 
> 
> Expected, really? I'm not all that familiar with floating point standards, but 
> I would have though this kind of thing would have been worked out long ago. 
> It's a pretty substantiate miscalculation. --And the fact that simply using 
> '*(1.0/0.1)' fixes it, makes me think so even more.

     My internal jury is still out.  Since 0.1 (base ten) in binary is a
repeating decimal (binimal?), there has to be some rounding assumptions
made somewhere.  By their very nature, such assumptions can't always be
right...but one might hope that they were never wrong in a case one
cared about.

     I'll think more about it, but may not come to any conclusions
before my one-week-in-Costa-Rica-with-no-internet.

-- Markus