[ruby-core:94296] [Ruby master Bug#10453] NUM2CHR() does not perform additional bounds checks

Issue #10453 has been updated by jeremyevans0 (Jeremy Evans).

File num2chr-range-check-10453.patch added

Attached is a patch that will add a range check to `NUM2CHR`.  However, it breaks a test:

```
1) Error:
TestStringIO#test_putc_nonascii:
RangeError: value to large to convert to char: 12356
    /home/jeremy/tmp/ruby/test/stringio/test_stringio.rb:567:in `putc'
    /home/jeremy/tmp/ruby/test/stringio/test_stringio.rb:567:in `test_putc_nonascii'
```

It is possibly, maybe even likely, that it will break third party C extensions as well (403 matches for `NUM2CHR` for Ruby on GitHub: https://github.com/search?l=Ruby&q=NUM2CHR&type=Code).

----------------------------------------
Bug #10453: NUM2CHR() does not perform additional bounds checks
https://bugs.ruby-lang.org/issues/10453#change-80644

* Author: silverhammermba (Max Anselm)
* Status: Assigned
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: ruby 2.1.4p265 (2014-10-27 revision 48166) [x86_64-linux]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN
----------------------------------------
`NUM2CHR()` just calls `rb_num2int_inline()` and masks off the high bytes. Consequently, passing any value larger than a `char` and no bigger than an `int` will return some garbage value (rather than raising `RangeError`).

To reproduce, compile and run:

~~~C
#include <ruby.h>
#include <limits.h>

int main(int argc, char* argv[])
{
    ruby_init();
    
    VALUE y = INT2FIX(INT_MAX);
    char z = NUM2CHR(y);
    
    printf("%hhd\n", z);
    
    return ruby_cleanup(0);
}
~~~

Expected:
Segfault from uncaught `RangeError`.

Actual:
Prints -1

---Files--------------------------------
num2chr-range-check-10453.patch (1.35 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>