Re: Is this a bug?

On Sep 20, 2010, at 18:00 , Gerry Jenkins wrote:
>> OK also inconsistent:
>>=20
>> irb(main):007:0> sprintf("%.0f",-1.5)
>> =3D> "-2"
>> irb(main):008:0> sprintf("%.0f",-10.5)
>> =3D> "-10"
>>=20
>> -1.5 rounds down to -2
>> -10.5 rounds up to -10
>=20
> fails even positive numbers:
>=20
> sprintf("%.0f",10.5)
> =3D> "10"
>=20
> irb(main):013:0> sprintf("%.0f",11.5)
> =3D> "12"

=
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conve=
rsion_and_rounding

"Rounding is used when the exact result of a floating-point operation =
(or a conversion to floating-point format) would need more digits than =
there are digits in the significand. There are several different =
rounding schemes (or rounding modes). Historically, truncation was the =
typical approach. Since the introduction of IEEE 754, the default method =
(round to nearest, ties to even, sometimes called Banker's Rounding) is =
more commonly used."

As for the original issue, how numbers ending in 0.05 are rounded for =
display with 1 digit after the decimal:

It does seem "inconsistent", but it isn't wrong.  Both 32.1 and 32.0 are =
equally distant from 32.05.  Internally, floating point numbers are =
represented as binary digits, so 32.05 isn't *exactly* 32.05, but the =
closest value that can be represented in binary, which will be ever so =
slightly higher or lower.

Try looking at the numbers with more decimal points and you'll see what =
I mean:

irb(main):043:0> sprintf("%.1f",-29.05)
=3D> "-29.1"
irb(main):044:0> sprintf("%.1f",-30.05)
=3D> "-30.1"
irb(main):045:0> sprintf("%.1f",-31.05)
=3D> "-31.1"
irb(main):046:0> sprintf("%.1f",-32.05)
=3D> "-32.0"
irb(main):047:0> sprintf("%.1f",-33.05)
=3D> "-33.0"
irb(main):048:0> sprintf("%.1f",-34.05)
=3D> "-34.0"
irb(main):049:0> sprintf("%.20f",-29.05)
=3D> "-29.05000000000000071054"
irb(main):050:0> sprintf("%.20f",-30.05)
=3D> "-30.05000000000000071054"
irb(main):051:0> sprintf("%.20f",-31.05)
=3D> "-31.05000000000000071054"
irb(main):052:0> sprintf("%.20f",-32.05)
=3D> "-32.04999999999999715783"
irb(main):053:0> sprintf("%.20f",-33.05)
=3D> "-33.04999999999999715783"
irb(main):054:0> sprintf("%.20f",-34.05)
=3D> "-34.04999999999999715783"

So what it's rounding isn't the exact number you're typing in, but the =
internal representation of that number in binary floating point format.

There are ways of tweaking the math so that the rounding seems more =
consistent, but no matter what, the values you enter won't be precisely =
equal to that value when you're using floating point numbers.

If you're not careful about rounding floating point digits, you can end =
up with something like this:

http://img.thedailywtf.com/images/200902/errord/DSC00669.JPG

Ben