[ruby-core:120354] [Ruby master Bug#20972] OpenSSL Memory Usage

Issue #20972 has been updated by byroot (Jean Boussier).


I think you are mostly correct, what exactly cause the memory growth is hard to point precisely (especially with `memory_profiler` because this codepath depends a lot on shared strings, so it's not fully clear which call exactly cause the unsharing.

But you are right that using `rb_str_locktmp` would avoid some allocations and probably some copying, and there are a few other small changes that can be made to rely less on string sharing, and reduce allocations a bit: https://github.com/ruby/openssl/pull/831

----------------------------------------
Bug #20972: OpenSSL Memory Usage
https://bugs.ruby-lang.org/issues/20972#change-111134

* Author: mrkgrandjean (Mark Grandjean)
* Status: Open
* ruby -v: 3.3.6
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
While testing large file uploads I noticed the OpenSSL Buffering is allocating a lot of memory for file uploads.
Similar to the issue in this ticket from a few years ago https://bugs.ruby-lang.org/issues/14426

Using the same test code from that ticket 
``` ruby
require "http"
require "memory_profiler"
require "stringio"

body = StringIO.new("a" * 5*1024*1024)

MemoryProfiler.report do
  HTTP.post("https://example.com", body: body)
end.pretty_print
```
in ruby 2.6.10

```
allocated memory by gem
-----------------------------------
   1913287  unicode_normalize
   1053463  llhttp-ffi-0.5.0
    187849  http-5.2.0
     50168  openssl
     36180  addressable-2.8.7
      8904  ipaddr
       272  other
```

While testing in Ruby 3.3.6

```
allocated memory by gem
-----------------------------------
  16104976  openssl
   2393456  unicode_normalize
   1053880  llhttp-ffi-0.5.0
    151633  http-5.2.0
      9813  addressable-2.8.7
      7872  ipaddr
       240  other
        40  forwardable

```

It looks like the memory allocation issue has returned. 



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/