[ruby-core:47606] [ruby-trunk - Bug #7037] float formatting inconsistently rounds half to even

Issue #7037 has been updated by headius (Charles Nutter).


The only definition I could find is a sideways use of "processor" to mean "Ruby processor", as in a Ruby language processor, i.e. a Ruby implementation. So... "the underlying platform of a conforming processor" would be the JVM, so it's probably ok for us to dodge the IEEE 754 issue.

Given the wording of the arithmetic rounding text in Float's description and the fact that formatting-related methods are not in the spec, we can probably dodge that issue too.

Of course, that's only if the ISO spec is all we care about :)

At this point I don't think I'm going to try to explore forcing strict IEEE 754, since even on the JVM it is intended as a workaround for specific, localized cases. So that leaves the rounding issue.

Given that there's no help from the spec on whether we should round half to even or half away from zero...what should we, as gentleman programmers, do?

If we agree to "half to even" then JRuby will round 1.05 to 1.0,  1.15 to 1.2, 1.85 to 1.8, and 1.95 to 2.0, which doesn't match MRI.

If we agree to "half away from zero" then MRI's rounding results of 1.25 to 1.2 would not match (the only other tiebreaking case in MRI's current system is 1.75, which already rounds to 1.8 as it would with "half away from zero").

And yes, I've seen that rounding code. I wish I could unsee it.
----------------------------------------
Bug #7037: float formatting inconsistently rounds half to even
https://bugs.ruby-lang.org/issues/7037#change-29559

Author: headius (Charles Nutter)
Status: Closed
Priority: Normal
Assignee: 
Category: core
Target version: 
ruby -v: 2.0.0dev


MRI does not appear to consistently round half to even. I'm not sure what rounding strategy this is, but it rounds xx05 and xx15 to odd for xxx, and other values to even:

irb(main):001:0> "%1.1f" % 1.05
=> "1.1"
irb(main):002:0> "%1.1f" % 1.15
=> "1.1"
irb(main):003:0> "%1.1f" % 1.25
=> "1.2"
irb(main):004:0> "%1.1f" % 1.35
=> "1.4"
irb(main):005:0> "%1.1f" % 1.45
=> "1.4"
irb(main):006:0> "%1.1f" % 1.55
=> "1.6"

None of the tie-breaking strategies I could find (http://en.wikipedia.org/wiki/Rounding#Tie-breaking) seem to support MRI's model.

If MRI is indeed using "half even", xx05 should round to xx0 and xx15 should round to xx2. An example with Java's BigDecimal appears to support this:

irb(main):029:0> java.math.BigDecimal.new('1.05').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.0"
irb(main):030:0> java.math.BigDecimal.new('1.15').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):031:0> java.math.BigDecimal.new('1.25').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.2"
irb(main):032:0> java.math.BigDecimal.new('1.35').round(java.math.MathContext.new(2, java.math.RoundingMode::HALF_EVEN)).to_s
=> "1.4"

We would like clarification about the proper rounding tie-breaker strategy to use so we can fix this JRuby issue properly: http://jira.codehaus.org/browse/JRUBY-6889


-- 
http://bugs.ruby-lang.org/