From: Peter Vandenabeele <peter@...>
Date: 2011-12-16T21:10:06+09:00
Subject: Re: Rounding error?

--00235446ff3ceaecdb04b4347a26
Content-Type: text/plain; charset=UTF-8

On Fri, Dec 16, 2011 at 12:51 PM, Jan Hendrickx <jan@vlaamsemolshoop.be>wrote:

> Hi there,
>
> I'm an absolute beginner in Ruby...
>
> Here's my problem:
>
> print("Length: ")
> s = gets()
> length = s.to_f
> print("Width: ")
> s = gets()
> width = s.to_f
> surface = length * width
> puts("Surface = #{length} x #{width} = #{surface}")
>
>
> When running this code, it asks for Length and Width as expected, but
> when I type the values 4.9 and 5.9 it gives 28.910000000000004 as
> result.
> The values 3.9 and 3.9 give 15.20999999999999 as result.
> What do I need to do to avoid these rounding-errors?
>

[Dutch: Dag Jan, welkom op de lijst]

Try to use BigDecimal.


http://www.ruby-doc.org/stdlib-1.9.3/libdoc/bigdecimal/rdoc/BigDecimal.html

BigDecimal.new("4.90") will create an "exact" fractional number in
the decimal system.

Take care that you feed a _string_ to BigDecimal:

Trying this

b = BigDecimal.new(4.9) would still cause the rounding error
so it is not supported.

When ready with calculations, use

big_decimal.to_s to convert back to a string.

A small demo:

peterv@e6500:~$ irb
ruby-1.9.3-p0 :001 > a = BigDecimal.new("4.90")
NameError: uninitialized constant BigDecimal
from (irb):1
from /home/peterv/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
ruby-1.9.3-p0 :002 > require 'bigdecimal'
 => true
ruby-1.9.3-p0 :003 > a = BigDecimal.new("4.90")
 => #<BigDecimal:946ad1c,'0.49E1',18(18)>
ruby-1.9.3-p0 :004 > b = BigDecimal.new(4.9)
ArgumentError: can't omit precision for a Rational.
from (irb):4:in `new'
from (irb):4
from /home/peterv/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
ruby-1.9.3-p0 :005 > c = BigDecimal.new("0.245")
 => #<BigDecimal:94238a4,'0.245E0',9(18)>
ruby-1.9.3-p0 :006 > d = a/c
 => #<BigDecimal:941d828,'0.2E2',9(45)>
ruby-1.9.3-p0 :007 > d.to_s
 => "0.2E2"
ruby-1.9.3-p0 :008 > d.to_s('F')
 => "20.0"

HTH,

Peter

-- 
Peter Vandenabeele
http://twitter.com/peter_v
http://rails.vandenabeele.com

--00235446ff3ceaecdb04b4347a26--