From: oinkoink+unet@... (Bret Jolly)
Date: 2004-05-20T13:53:48+09:00
Subject: Re: bad maths

Martin larsson <morg@borgeby.se> wrote in message news:<40ABE654.7070601@borgeby.se>...
> Martin larsson wrote:
[repeated adding of 0.1 not coming out exact]

The problem is that the computer thinks in binary and
0.1 has an infinitely long binary representation. (You
won't see this "problem" if you instead add 0.5, which
has a finitely long representation)  This is not a ruby
problem, but a limitation of the computer's floating point.
In general, you should regard floating point numbers as
being slightly inexact.  If you need exactness, require
the mathn library and use rationals.  Imitating your example,
you'd get something like:

require 'mathn'
i = 5
11.times {puts i += 1/10}  # => exact rational results

As an alternative, you could limit the precision of your
output, which effectively rounds the numbers to output
precision:

i = 5.5
5.times { puts "%5.3f" % (i += 0.1)}
#output follows:
5.600
5.700
5.800
5.900
6.000

Regards, Bret
Bret Jolly (Jou Lii Bair)