[ruby-core:53969] Re: [ruby-trunk - Bug #7829] Rounding error in Ruby Time

On Apr 3, 2013, at 5:15 AM, Tanaka Akira wrote:

> 2013/2/22 David MacMahon <[e-mail]>:
> 
>> What do people thing about changing num_exact() in time.c to call #rationalize for Floats rather than #to_r?  Or perhaps call #rationalize on the object if it responds to #rationalize so that this won't be exclusive to Floats?
> 
> I'm not sure Float#rationalize is good choice.
> At least, I don't understand the behavior and
> the document don't explain the behavior.
> The document describes eps is choosen automatically.
> I think it is not enough explanation.

I agree that the documentation is not explicit on how eps is chosen automatically in Float#rationalize.

My assumption has been that eps is chosen such that the resulting Rational will convert to the exact same double precision value that was originally given.  In other words, "f.rationalize.to_f == f" will always be true assuming #rationalze doesn't raise FloatDomainError (e.g. if f is NaN or Infinity).

I have assumed that Float#rationalize is based on the same concept as the atod function described in this paper:

http://www.ampl.com/REFS/rounding.pdf

Based on some quick tests, it seems like my assumptions are wrong (as assumptions often are!) at least on "ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]".  My assumptions about Float#rationalize do not hold for small (absolute) values around and below 1.0e-17.  String#to_r has even more problem cases.  Here are two examples:

>> f=57563.232824357045
=> 57563.232824357045

>> puts "%016x\n"*5 % [f, f.to_r.to_f, f.to_s.to_f, f.to_s.to_r.to_f, f.rationalize.to_f].pack('D*').unpack('Q*')
40ec1b67734c10e7
40ec1b67734c10e7
40ec1b67734c10e7
40ec1b67734c10e6 <=== String#to_r "error"
40ec1b67734c10e7
=> nil

>> f=1e-17
=> 1.0e-17

>> puts "%016x\n"*5 % [f, f.to_r.to_f, f.to_s.to_f, f.to_s.to_r.to_f, f.rationalize.to_f].pack('D*').unpack('Q*')
3c670ef54646d497
3c670ef54646d497
3c670ef54646d497
3c670ef54646d497
3c670ef54646d498 <=== Float#rationalize "error"
=> nil

I regard the String#to_r error to be a bug (i.e unintended and undesirable behavior).  I find the Float#rationalize error to be undesirable behavior (IMHO), but since the documentation is vague I can't really say that it is unintended behavior.

In both cases, however, the error is very small (just one LSb of the mantissa).

Dave