From: naruse@... Date: 2017-12-01T13:23:37+00:00 Subject: [ruby-core:84027] [Ruby trunk Bug#5273][Open] Float#round returns the wrong floats for higher precision Issue #5273 has been updated by naruse (Yui NARUSE). Status changed from Assigned to Open Assignee deleted (naruse (Yui NARUSE)) ---------------------------------------- Bug #5273: Float#round returns the wrong floats for higher precision https://bugs.ruby-lang.org/issues/5273#change-68110 * Author: marcandre (Marc-Andre Lafortune) * Status: Open * Priority: Normal * Assignee: * Target version: next minor * ruby -v: r33186 * Backport: 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- Float#round returns the wrong floats for higher precision. Rounding can be off: 2.5e-22.round(22) # => 2.0e-22 It can also return values that have extra decimals: 2.5e-31.round(31) # => 3.0000000000000003e-31 2.5e-36.round(36) # => 2.9999999999999998e-36 Both errors can occur at the same time too: 2.5e-39.round(39) # => 2.0000000000000002e-39 I believe these errors occur only for ndigits >= 22. For ndigits > 22, 10**(-ndigits) is not an exact Float (as 5 ** 23 won't fit in the 53 bits of a double's mantissa). For ndigits < 22, there should be enough bits left to avoid rounding errors. For ndigits >= 22, the algorithm to use should be similar to the one used to convert doubles to string. Hopefully, this is the last bug for #round with a given precision. -- https://bugs.ruby-lang.org/ Unsubscribe: