[ruby-core:30145] [Bug #3273] Float string conversion

Bug #3273: Float string conversion
http://redmine.ruby-lang.org/issues/show/3273

Author: Marc-Andre Lafortune
Status: Open, Priority: Normal
Assigned to: Marc-Andre Lafortune, Category: core, Target version: 1.9.2
ruby -v: ruby 1.9.3dev (2010-05-11 trunk 27730) [x86_64-darwin10.3.0]

For any float f, the two following conditions should hold:
(1) f.to_s.to_f == f        (round trips)
(2) f.to_s.chop.to_f != f   (minimal)

The second condition is a simplification; if the string representation is in scientific notation, than the character to remove would be the one just before the "e". Also, if the string representation ends with ".0", then it is minimal.

Currently, the first condition fails in Ruby 1.8, and the second fails in Ruby 1.9

  $ ruby18dev -ve 'f = 0.21611564636388508; puts f.to_s.to_f == f'
  ruby 1.8.8dev (2010-05-11) [i386-darwin10.3.0]
  false

  $ rubydev -ve 'f = 0.56; puts f.to_s.chop.to_f != f'
  ruby 1.9.3dev (2010-05-11 trunk 27730) [x86_64-darwin10.3.0]
  false

Note that this implies that Ruby 1.8 and 1.9 do not output the same string representation for either of these two floats.

The conversion algorithm currently checks two precisions. In Ruby 1.9, it tries 16 digits and if that's not enough it then uses 17. In 1.8, it's the same but with 15 and 16.

The fact is that 17 can be necessary (e.g. 0.21611564636388508 is not equal to either 0.2161156463638851 or 0.2161156463638850) and 16 can be too much (e.g. 0.5600000000000001 == 0.56), so three precisions must be checked.

The following patch fixes this issue for trunk (although it can probably be made nicer and/or faster).

Let me know if there are any objections to fixing both the 1.9 and 1.8 lines.


diff --git a/numeric.c b/numeric.c
index f2c8c13..442b069 100644
--- a/numeric.c
+++ b/numeric.c
@@ -569,7 +569,8 @@ flo_to_s(VALUE flt)
     else if (isnan(value))
        return rb_usascii_str_new2("NaN");
 
-# define FLOFMT(buf, size, fmt, prec, val) snprintf(buf, size, fmt, prec, val), \
+# define FLOFMT(buf, size, fmt, prec, val) snprintf(buf, size, fmt, prec-1, val), \
+    (void)((atof(buf) == val) || snprintf(buf, size, fmt, (prec), val)), \
     (void)((atof(buf) == val) || snprintf(buf, size, fmt, (prec)+1, val))
 
     FLOFMT(buf, sizeof(buf), "%#.*g", float_dig, value); /* ensure to print decimal point */


----------------------------------------
http://redmine.ruby-lang.org