From: cilibrar@... (Rudi Cilibrasi)
Date: 2003-04-17T02:13:33+09:00
Subject: Roundoff problem with Float and Marshal

The following small test program:
[1.5, 4.0/3.0].each { |a|
ma = Marshal.dump(a)
b = Marshal.load(ma)

if a == b
  puts "Everything is working fine for #{a}"
else
  puts "PROBLEM: a is #{a}, b is #{b}, and ma is #{ma.dump}"
end
}

Exposes the following strange behavior:
        ruby test.rb
Everything is working fine for 1.5
PROBLEM: a is 1.33333333333333, b is 1.33333333333333, and ma is
"\004010f\0261.333333333333333"

So apparently Marshal *sometimes* changes the value of a 
Float when dumped.  The problem looks like it is caused by 
an attempt to make a platform-independent Marshal format 
using an ASCII-decimal like representation, which cannot 
accurately express all possible double's. I understand that
this is an impossible problem since different floating point
formats have different sets of expressible points on the real
line. I think it would be more convenient and useful in 
scientific computing, however, if we could exactly save and 
load floating-point values when we are on a consistent 
platform like the IEEE 754 standard, and just want to use 
Marshal to save/load on disk or across a network with no
numerical instability.  Here is one way to achieve it using a
special byte:

Choose the three most popular floating-point formats, and 
give them codes, like 
0 = IEE754x86, 1 = SPARC, 2 = MAC68k, 3 = OTHER
Then add a single byte to the end of the Marshal string that 
bitwise copies in a platform-dependent format the 
6 least-significant bits of the significand, wherever they 
may be according to the spec.  Then, when loading up a Float, 
first read the decimal expansion as usual to get a "close" 
value.  Next, check the 2-bit format code to see if the native
Float format is the same. If it is, copy over the six bits of
platform-dependent significand over the close value.  Now it's 
precisely right when the format is the same, and "pretty 
close" in other cases.  If we're concerned about the size of the 
Marshal format, then it would probably do as well to simply 
replace the last decimal digit of a maximum-sized decimal 
expansion with this special byte, as it contains more information 
than a single decimal digit. Then, this special byte would only
incur an extra cost in the relatively small fraction of Float's
that have submaximal decimal expansions.  Finally, it's not 
hard to see how a single bit can be used instead to indicate 
the presence or abscence of the special byte, to further cut 
down on space used.

This technique would allow me to do long scientific simulations 
that have to load and save lots of intermediate files and over 
many different computers working in parallel and communicating 
results via network.

What do you all think?

Rudi
(another ruby hack)