[ruby-core:104669] [Ruby master Bug#18018] Float#floor / truncate sometimes result that is too small.

Issue #18018 has been updated by jeremyevans0 (Jeremy Evans).

Status changed from Open to Feedback

The algorithm for `Float#floor` with the `ndigits` argument is basically (omitting the overflow/underflow handling):

```ruby
class Float
  def floor_ndigits(ndigits)
    x = (10**ndigits).to_f
    (self * x).floor / x
  end
end
```

Let's see what `self * x` is in each case:

```ruby
291.4 * 10      # 2914.0
291.4 * 100     # 29139.999999999996
291.4 * 1000    # 291400.0
291.4 * 10000   # 2914000.0
291.4 * 100000  # 29139999.999999996
291.4 * 1000000 # 291400000.0
```

This issue also goes the other direction:

```
f = 291.39999999999997
6.times.map{|i| f.floor(i)}
# => [291, 291.4, 291.39, 291.4, 291.4, 291.39999]
```

`Float#floor` results are inexact, because `Float` itself is inexact, so I don't think the current behavior is a bug.

marcandre (Marc-Andre Lafortune) wrote:
> `g = f.floor(n)`, for `n > 0` must return the highest float that has the correct properties:
> * `g` <= `f`
> * `g`'s decimal string representation has at most `n` digits

I think these are both true in these cases. 291.4, 291.39, and 219.39999 are all <= 291.4, and the decimal string representation has at most the number of digits specified after the decimal point.

> I'll note that `floor` should be stable, i.e. `f.floor(n).floor(n) == f.floor(n)` for all `f` and `n`.

This is also true:

```ruby
291.4.floor(2).floor(2) == 291.4.floor(2) # true
291.4.floor(5).floor(5) == 291.4.floor(5) # true
```

@marcandre If you still think this is a bug, could you explain why, and ideally the algorithm that should be used instead?

----------------------------------------
Bug #18018: Float#floor / truncate sometimes result that is too small.
https://bugs.ruby-lang.org/issues/18018#change-92982

* Author: marcandre (Marc-Andre Lafortune)
* Status: Feedback
* Priority: Normal
* Target version: 3.1
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------

```ruby
291.4.floor(1) # => 291.4 (ok)
291.4.floor(2) # => 291.39 (not ok)
291.4.floor(3) # => 291.4 (ok)
291.4.floor(4) # => 291.4 (ok)
291.4.floor(5) # => 291.39999 (not ok)
291.4.floor(6) # => 291.4 (ok)
```

`g = f.floor(n)`, for `n > 0` must return the highest float that has the correct properties:
* `g` <= `f`
* `g`'s decimal string representation has at most `n` digits

I'll note that `floor` should be stable, i.e. `f.floor(n).floor(n) == f.floor(n)` for all `f` and `n`.

Same idea for `truncate`, except for negative numbers (where `(-f).truncate(n) == -(f.floor(n))` for positive `f`).

Noticed by Eust痃uio Rangel but posted on the mailing list.

Please do not reply that I need to learn how floats work. Note that example given in doc `(0.3/0.1).floor == 2` is not this issue, since `0.3/0.1 #=> 2.9999999999999996`



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>