[ruby-core:120159] [Ruby master Bug#20937] "can't set length of shared string" error when using OpenSSL::Cipher#update with buffer

Issue #20937 has been updated by byroot (Jean Boussier).


Ah indeed.

It can be fixed on the openssl side with:

```ruby
diff --git a/ext/openssl/ossl_cipher.c b/ext/openssl/ossl_cipher.c
index 5a491d8..2e87c60 100644
--- a/ext/openssl/ossl_cipher.c
+++ b/ext/openssl/ossl_cipher.c
@@ -414,6 +414,7 @@ ossl_cipher_update(int argc, VALUE *argv, VALUE self)
     if (!ossl_cipher_update_long(ctx, (unsigned char *)RSTRING_PTR(str), &out_len, in, in_len))
    ossl_raise(eCipherError, NULL);
     assert(out_len <= RSTRING_LEN(str));
+    rb_str_modify(str);
     rb_str_set_len(str, out_len);
 
     return str;
```

But in a way I find it weird that `rb_str_set_len` wouldn't make the string independant if it has to, so I think it could be patched on the Ruby side too.

----------------------------------------
Bug #20937: "can't set length of shared string" error when using OpenSSL::Cipher#update with buffer
https://bugs.ruby-lang.org/issues/20937#change-110914

* Author: akiellor (Andrew Kiellor)
* Status: Open
* Assignee: rhenium (Kazuki Yamaguchi)
* ruby -v: ruby 3.3.6 (2024-11-05 revision 75015d4c1f) +YJIT [arm64-darwin23]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
We've observed a recurring "can't set length of shared string" error in production emerging from the `aws-sdk-s3` library when using it's client encryption features. The sdk in this mode uses OpenSSL::Cipher in decrypt mode with a String buffer. It appears that under some circumstances the buffer becomes a "shared string" and is no longer compatible with the requirements of OpenSSL::Cipher#update.

I've attached a reproduction scenario using only the ruby standard library.

``` shell
$ ruby -v
ruby 3.3.6 (2024-11-05 revision 75015d4c1f) +YJIT [arm64-darwin23]
$ ruby scripty.rb
scripty.rb:32:in `update': can't set length of shared string (RuntimeError)
	from scripty.rb:32:in `block (2 levels) in <main>'
	from scripty.rb:31:in `each'
	from scripty.rb:31:in `block in <main>'
	from /nix/store/fhmbmmzr4h5yax66nl2x44rrdf039b3s-ruby-3.3.6/lib/ruby/3.3.0/tempfile.rb:447:in `create'
	from scripty.rb:26:in `<main>'
```

The attached script has a few different read patterns, some fail and some do not.

Credit for isolating this issue goes to [@simoleone](https://bugs.ruby-lang.org/users/55425).

---Files--------------------------------
scripty.rb (1.15 KB)


-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/