[ruby-core:81370] [Ruby trunk Misc#13597] Does read_nonblock call remalloc for the buffer if does it just set the size attribute

Issue #13597 has been reported by emilys (Emily Stolfo).

----------------------------------------
Misc #13597: Does read_nonblock call remalloc for the buffer if does it just set the size attribute
https://bugs.ruby-lang.org/issues/13597

* Author: emilys (Emily Stolfo)
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
Hello

I've observed that a lot of memory gets allocated and wasted when read_nonblock is called for a number of bytes much larger than is actually read from the socket.
This line https://github.com/ruby/ruby/blob/0130bb01baed404c0e3c75bd5db472415a6da1d3/io.c#L2686 appears to eventually only change the heap size value here https://github.com/ruby/ruby/blob/144e06700705a3f067582682567bc77b429c4fca/string.c#L104 but does not call remalloc.

I see this request to allow an offset to be passed to read_nonblock:
https://bugs.ruby-lang.org/issues/11484

but until that is implemented, how do you recommend efficiently asking to read a large number of bytes from a socket? If I'm not mistaken, if I request 16000000, but only read 1000000, the buffer that has been allocated in io_read_nonblock for 16000000 doesn't seem to be resized.

Would you recommend instead requesting a more predictable number of bytes, closer to the default system value (SO_RCVBUF, for example) in each call to read_nonblock?

For context, this pull request against the MongoDB Ruby driver has lead me to this investigation. https://github.com/mongodb/mongo-ruby-driver/pull/864

Thank you in advance
Emily





-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>