From: Eric Wong Date: 2018-05-16T10:07:07+00:00 Subject: [ruby-core:87077] Re: [Ruby trunk Bug#14745] High memory usage when using String#replace with IO.copy_stream janko.marohnic@gmail.com wrote: > I wrote: > > Finally, I always assumed your example is a contrived case and > > you're dealing with an interface somewhere (not StringIO) which > > doesn't accept a destination buffer for .read. > The example was simplified for reproducing purposes. The place > where I discovered this was in > https://github.com/rocketjob/symmetric-encryption/pull/98 (I > eventually managed to figure out `String#replace` was causing > the high memory usage, so I switched to `String#clear`). > > In short, the `SymmetricEncryption::Reader` object wraps an IO > object with encrypted content, and when calling `#read` it > reads data from the underlying IO object, decrypts it and > returns the decrypted data. So, it's not patching the lack of > outbuf argument (because the underlying IO object *should* > accept the outbuf argument), rather it provides an `IO#read` > interface over incrementally decrypting IO object content. I may be misreading(*), but it looks like @stream_cipher.update can take a second destination arg (like IO#read and friends) and maybe that helps... (that appears to be OpenSSL::Cipher#update) If that's not usable somehow, I've sometimes wondered if adding a String#exchange! method might help: foo = "hello............................" bar = "world............................" foo.exchange!(bar) p foo # => "world............................" p bar # => "hello............................" In your case, you could still use String#clear with this: dst.clear dst.exchange!(tmp) # tmp is now '' because dst was cleared before For non-embed strings, it would swap the pointers and be zero-copy, but won't suffer the new-frozen-string-behind-your-back problem of String#replace. (*) I'm looking at your commit 8d41efacff26f3357016d6b611bee174802fba66 after git clone-ing (since I don't do JS-infested web UIs) Unsubscribe: