Re: Roundoff problem with Float and Marshal

Nobu writes:
>> The important point would be that the least significant floating-point
>> mantissa 8 bits are put in the first byte, then the next least significant
>> 6 bits in the second byte.  This is endian-safe so long as you put them
>> back right, and are using IEEE754.
>
>Sorry, but I can't understand why you call it endian-safe.  It
>seems to definitely depend on the endian.

Firstly, I hope you are looking at the version of the patch that is most
recent, that Guy Decoux responded to, using the GETBYTE macro.

This patch works on big-endian and little-endian machines.  Guy Decoux
tried my test program on both using this patch and it reports no problems;
(he did not post this test to the newsgroup, but communicated this to
me via private email when I asked him to run my test program; it produces
no failure cases, that's why he had to create a new test program)
therefore, the patch successfully solves the problem of having different
numbers come back from Marshalling, on both an x86 and a Sun so long
as you run my original program.  If you try it, you will also see that
it works fine when you load and store from little-endian to big-endian
machines, for instance x86 to Sun.  In fact, Guy Decoux did just this,
and you can see that he had to make it repeat 32 times to find a
single error.  That means that 31 out of 32 times, both the Sun
version and the x86 version of his program produced exactly the same
results as saved in his file.  If the patch were actually not endian-safe, 
then this would have failed (practically) every time.  Instead, we see it
working almost all the time, and my explanation for it not being 100% accurate
when going from x86 to Sun is that the underlying calculations themselves, 
such as Math.exp, floating point addition, etc, are implemented differently.
It is not from Marshalling inaccuracies, which have been fixed by this.
If you look at my patch, you can see that it checks what type of
IEEE754 is in use at runtime, and sets floatingStart and floatingDir
accordingly; either (0,1) or (7,-1) depending on if the endianness
is little or big, respectively.  Then, every time I access a byte in
the double, I do it with respect to these constants (using the GETBYTE
macro which accesses bytes starting with floatingStart going +/-) thus it
is endian-independent.  In other words, the first byte after the
terminating NUL in the marshalled string is always the least significant
8 bits of the mantissa, regardless of whether the machine is little
or big endian floats.  The next byte is the next 6 bits of the mantissa,
which are more significant than the other 8, again regardless of the
endianness of the machine.  This is just like how the htons function
works to create an endian-independent format for shorts.  You could
say I store the mantissa of the float in little-endian format.  When reading
these bytes back, I again adjust for the endianness appropriately, so
that nothing bad occurs.

If you still think that the patch is endian unsafe, then it should not be
hard to come up with a test program that does one of two things:

1) on a single machine, shows that a number that is written out using
Marshal.dump is not the same as the one read back in using Marshal.load.

2) on a pair of machines, shows that a number written on one using
Marshal.dump to a file is read back in incorrectly using Marshal.load
on the same file on a different machine.  Note that Guy Decoux was probably
trying to demonstrate this effect using his test program, but in fact
I believe he demonstrated instead that the standard floating point math
functions are not 100% consistent across platforms (as the spec allows
some latitude), and in fact indirectly proved that the patch is working
in an endian-independent fashion.

Does this make sense?  If not please let me clarify further as this makes
a big difference in my use (and particularly testing of math functions)
of Ruby, as many test cases for loading and saving more complicated objects
such as Complex numbers must rely on the basic loading and saving of
Marshal working correctly for floats.

Happy Easter,

Rudi