From: Markus <markus@...>
Date: 2004-10-16T01:02:51+09:00
Subject: Re: Rounding to X digits

On Fri, 2004-10-15 at 07:28, trans. (T. Onoma) wrote:
> On Thursday 14 October 2004 01:55 pm, Markus wrote:
> | Here's mine (with a few extras):
> |
> | class Numeric
> |     def to_the_nearest(n)
> |         n*(self/n).round
> |         end
> |     end
> |
> 
>   def round_to_nearest( n=0.01 )
>     (self * (1/n)).round.to_f / (1/n)
>   end
> 
> For some reason they don't give the exact same answer. Try an edge case.
> 
>   irb(main):066:0> 0.1 * (134.45/0.1).round
>   => 134.4
>   irb(main):067:0> (134.45 * (1/0.1)).round.to_f / (1/0.1)
>   => 134.5

     Peeling one layer of the onion, they differ because:

irb(main):033:0> printf "%20.15f",(134.45 * (1.0/0.1))
1344.500000000000000=> nil
irb(main):034:0> printf "%20.15f",(134.45/0.1)
1344.499999999999773=> nil

I suppose this is not unexpected (my mama warned me 'bout floats) but it
is a little unexpected--no, I'm wrong, 1/5 is a repeating decimal base
2, so it's perfectly expected.  The tricky bit is, which form (if either
of them) will always (or at least, more generally) give the correct
result?  I can't see off hand that either will be intrinsically "better"
but I could be missing a point.  I suspect (SWAG) that the hybrid:

     def to_the_nearest(n)
         if self.abs < 1.0 
             (self/n).round/n
           else
             m = 1.0/m
             (self*m).round/m
           end
        end

would do better than both, but I haven't tested it.

-- Markus

P.S. Mine has mostly been tested on values < 1.0, thus my suspicion that
it works well in that domain.  Typing this though, I realize that it was
tested FOR CONFORMANCE WITH A PRE-EXISTING SYSTEM* which itself may have
been buggy.

* Think 25 year old spaghetti FORTRAN, then sigh in bliss realizing that
your imagination is much nicer than the ugly facts.