From: Luis Parravicini <lparravi@...>
Date: 2007-08-10T04:06:31+09:00
Subject: precision loss with pack/unpack ?

Hello

I'm reading Java class files using ruby and found some problems when
handling float values. The numbers are stored using IEEE 754 format.
After several hours trying and thinking I might be using unpack the
wrong way, I've wrote the following code which packs a Float and then
immediately unpacks it. And the result is different from the original
value.

------------
a = 0.1234567
b = [a].pack('g').unpack('g')[0]

puts "#{a} =? #{b} => #{a == b}"
------------

outputs:

0.1234567 =? 0.123456701636314 => false

I came accross this doc, which states that in Perl this happens too
(http://perldoc.perl.org/functions/pack.html) "Note that Perl uses
doubles internally for all numeric calculation, and converting from
double into float and thence back to double again will lose precision
(i.e., unpack("f", pack("f", $foo)) will not in general equal $foo)."
Maybe Ruby is doing the same Perl does?



Any help will be appreciated

Thanks


-- 
Luis Parravicini
http://ktulu.com.ar/blog/