From: Yossef Mendelssohn <ymendel@...>
Date: 2007-10-23T22:33:24+09:00
Subject: Re: computers + floating-point = a chore

On Oct 22, 10:14 pm, Gary Wright <gwtm...@mac.com> wrote:
> On Oct 22, 2007, at 10:30 PM, Yossef Mendelssohn wrote:
>
> > Yes, I know the troubles with floating point. I think what I found
> > most interesting was a number that claimed to be 19.76 when it
> > actually wasn't, simply because it's more convenient (?) to display it
> > as such.
>
> Well, the culprit is actually Float#to_s, which has it's own idea of
> how to convert floating point numbers to text.  In particular it
> defaults
> to 15 digits of precession and also works hard to strip trailing zero's:
>
> $ irb
> irb(main):001:0> a = 19.76
> => 19.76
> irb(main):002:0> a.to_s
> => "19.76"
> irb(main):003:0> sprintf "%#.15g", a
> => "19.7600000000000"
> irb(main):004:0> sprintf "%#.32g", a
> => "19.760000000000001563194018672220"
> irb(main):005:0>
>
> You can see from this session that 15 digits of precision is not
> enough to 'show' that a isn't really 19.76.
>
> What was surprising to me is that Float#to_s doesn't just use a standard
> sprintf format string:
>
> static VALUE
> flo_to_s(flt)
>      VALUE flt;
> {
>      char buf[32];
>      double value = RFLOAT(flt)->value;
>      char *p, *e;
>
>      if (isinf(value))
>          return rb_str_new2(value < 0 ? "-Infinity" : "Infinity");
>      else if(isnan(value))
>          return rb_str_new2("NaN");
>
>      sprintf(buf, "%#.15g", value); /* ensure to print decimal point */
>      if (!(e = strchr(buf, 'e'))) {
>          e = buf + strlen(buf);
>      }
>      if (!ISDIGIT(e[-1])) { /* reformat if ended with decimal point
> (ex 111111111111111.) */
>          sprintf(buf, "%#.14e", value);
>          if (!(e = strchr(buf, 'e'))) {
>              e = buf + strlen(buf);
>          }
>      }
>      p = e;
>      while (p[-1]=='0' && ISDIGIT(p[-2]))
>          p--;
>      memmove(p, e, strlen(e)+1);
>      return rb_str_new2(buf);
>
> }
>
> Gary Wright

Thanks for this, Gary. I wasn't expecting that Float#inspect would
call #to_s, and obviously didn't even bother to test anything out with
sprintf.

This also explains another Float niggle, which is why it will display
"Infinity" but trying to use Infinity as a constant doesn't work.

Maybe I should spend more time looking at the Ruby source.

--
-yossef