[ruby-core:67580] [ruby-trunk - Bug #10633] OpenSSL RangeError on update

Issue #10633 has been updated by Usaku NAKAMURA.

Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE to 2.0.0: DONE, 2.1: REQUIRED, 2.2: DONE

Backported into `ruby_2_0_0` at r49249.

----------------------------------------
Bug #10633: OpenSSL RangeError on update
https://bugs.ruby-lang.org/issues/10633#change-50994

* Author: Kenneth Hoffmann
* Status: Closed
* Priority: Normal
* Assignee: openssl
* ruby -v: 2.1.3p242 (2014-09-19 revision 47630) [x86_64-linux]
* Backport: 2.0.0: DONE, 2.1: REQUIRED, 2.2: DONE
----------------------------------------
OpenSSL Cipher throws a RangeError on the update function when given a very large string. I've verified that it happens with a file of 2 or 3GB in size. 1 GB works fine. This has been tested with 2.1.2, 2.1.3, and 2.2rc-1.

```ruby
require 'openssl'
cipher = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
cipher.decrypt
cipher.iv = "fooooooooooooooooooooooooooooooooo"
cipher.key = "barrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr"
f = File.open("<path to a 2 GB file generated with dd>", "r")
contents = f.read;
cipher.update(contents)
```

Running this will return:

```
RangeError: integer 2147483648 too big to convert to `int'
```

File generated with dd:

```
dd if=/dev/zero of=2g.img bs=1 count=0 seek=2G
```




-- 
https://bugs.ruby-lang.org/