Re: Is this a bug?

Thank you Ben,

I did discover that there are various ways of rounding just before your
reply at  http://en.wikipedia.org/wiki/Rounding

I am writing a ruby web based unit testing for my java students to 
submit their programs to.

My real problem is that sprintf in ruby behaves differently in this 
regard to how java rounds.

Oh well, I fixed it by creating unit test data that was within the 
precision of the final java output, so that rounding was not used in the 
unit testing.

Ben Giddings wrote:
> On Sep 20, 2010, at 18:00 , Gerry Jenkins wrote:
>> fails even positive numbers:
>> 
>> sprintf("%.0f",10.5)
>> => "10"
>> 
>> irb(main):013:0> sprintf("%.0f",11.5)
>> => "12"
> 
> http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
> 
> "Rounding is used when the exact result of a floating-point operation 
> (or a conversion to floating-point format) would need more digits than 
> there are digits in the significand. There are several different 
> rounding schemes (or rounding modes). Historically, truncation was the 
> typical approach. Since the introduction of IEEE 754, the default method 
> (round to nearest, ties to even, sometimes called Banker's Rounding) is 
> more commonly used."
> 
> As for the original issue, how numbers ending in 0.05 are rounded for 
> display with 1 digit after the decimal:
> 
> It does seem "inconsistent", but it isn't wrong.  Both 32.1 and 32.0 are 
> equally distant from 32.05.  Internally, floating point numbers are 
> represented as binary digits, so 32.05 isn't *exactly* 32.05, but the 
> closest value that can be represented in binary, which will be ever so 
> slightly higher or lower.
> 
> Try looking at the numbers with more decimal points and you'll see what 
> I mean:
> 
> irb(main):043:0> sprintf("%.1f",-29.05)
> => "-29.1"
> irb(main):044:0> sprintf("%.1f",-30.05)
> => "-30.1"
> irb(main):045:0> sprintf("%.1f",-31.05)
> => "-31.1"
> irb(main):046:0> sprintf("%.1f",-32.05)
> => "-32.0"
> irb(main):047:0> sprintf("%.1f",-33.05)
> => "-33.0"
> irb(main):048:0> sprintf("%.1f",-34.05)
> => "-34.0"
> irb(main):049:0> sprintf("%.20f",-29.05)
> => "-29.05000000000000071054"
> irb(main):050:0> sprintf("%.20f",-30.05)
> => "-30.05000000000000071054"
> irb(main):051:0> sprintf("%.20f",-31.05)
> => "-31.05000000000000071054"
> irb(main):052:0> sprintf("%.20f",-32.05)
> => "-32.04999999999999715783"
> irb(main):053:0> sprintf("%.20f",-33.05)
> => "-33.04999999999999715783"
> irb(main):054:0> sprintf("%.20f",-34.05)
> => "-34.04999999999999715783"
> 
> So what it's rounding isn't the exact number you're typing in, but the 
> internal representation of that number in binary floating point format.
> 
> There are ways of tweaking the math so that the rounding seems more 
> consistent, but no matter what, the values you enter won't be precisely 
> equal to that value when you're using floating point numbers.
> 
> If you're not careful about rounding floating point digits, you can end 
> up with something like this:
> 
> http://img.thedailywtf.com/images/200902/errord/DSC00669.JPG
> 
> Ben

-- 
Posted via http://www.ruby-forum.com/.