[ruby-core:54033] Re: [ruby-trunk - Bug #7829] Rounding error in Ruby Time

On Apr 4, 2013, at 7:02 PM, Tanaka Akira wrote:

> It is expected that Rational#to_f can error because Float has only
> 53bit mantissa but Rational can hold more digits.
> (Ruby uses double type in C and it is usally IEEE 754 double which has
> 53bit mantissa.)

I understand that the Float returned by Rational#to_f has limited precision and often will only approximate but not equal the value represented by the Rational.  But in the example of 57563.232824357045 we are talking about a Float value that is representable.  I think it is reasonable to expect f.to_r.to_f == f.  I think that this is possible, but it requires changing both Float#to_r and Rational#to_f and I do not have a sense of whether it is practical from a performance point of view.

Float#to_r effectively converts the sign and binary mantissa of the Float to Rational and then either multiplies it by 2**exponent or divides it by 2**(-exponent) if exponent is negative.  This creates a Rational which accurately represents the value represented by the bits underlying the Float.  IOW, it rationalizes the binary approximation represented by the Float rather than the corresponding decimal approximation of the Float (which is what f.to_s.to_r does).  IMHO, it would be better to rationalize the decimal approximation as these examples show:

>> 1e23.to_r
=> (99999999999999991611392/1) <=== That's not exactly 1e23

But luckily it rounds back to the original Float:

>> 1e23.to_r.to_f
=> 1.0e+23

Unfortunately, doing math on it can bring us "bad luck":

>> (1e23.to_r/100).to_f
=> 999999999999999900000.0 <=== Should be 1.0e+21

It's only differs by the least significant bit of the mantissa, but doesn't follow the principle of least surprise.  Converting the Float to Rational via String (i.e. rationalizing the decimal approximation) avoids this issue:

>> (1e23.to_s.to_r/100).to_f
=> 1.0e+21

While changing Float#to_r to do the equivalent of "self.to_s.to_r" leads to "better" (can you find any counter examples?) rationalizations, it does not deal with rounding issues when converting to Float.  The current Rational#to_f converts numerator and denominator to Floats then divides them.  This results in three potential roundings: one for numerator to Float, one for denominator to Float, and one for the quotient.  Using higher than double precision internally (e.g. via BigDecimal) and then rounding only at the end when converting to Float will lead to higher quality results as this example (again) shows:

>> 57563.232824357045.to_s.to_r.to_f
=> 57563.23282435704

>> require 'bigdecimal'; require 'bigdecimal/util'
=> true

>> 57563.232824357045.to_s.to_r.to_d(18).to_f
=> 57563.232824357045

The only Floats I have found for which f.to_s.to_r.to_d(18).to_f == f does NOT hold are subnormals and I think that is exposing a bug in BigDecimal#to_f:

>> 2.58485e-319.to_s.to_r.to_d(18)
=> #<BigDecimal:2692588,'0.258485E-318',9(45)>

>> 2.58485e-319.to_s.to_r.to_d(18).to_f
=> Infinity

Maybe this is fixed in newer versions.  I am running "ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]".

> It seems your requirement is too strong for Float.

I think having Float#to_r represent the decimal approximation of the Float would lead to less surprise.  Until someone creates a patch and it is accepted, this can be accomplished by monkey patching Float#to_r (though performance may suffer).

I think having Rational#to_f use higher precision internally would lead to higher precision results.  This could also be accomplished via monkey patching, perhaps as part of bigdecimal.

Dave