From: me@... Date: 2020-01-24T15:20:25+00:00 Subject: [ruby-core:96989] [Ruby master Bug#16559] Net::HTTP#request does not properly close TCP socket if #started? is false Issue #16559 has been updated by f3ndot (Justin Bull). ruby -v changed from 2.7.0-preview to 2.8.0-dev, 2.7.0, 2.6.5 File dont-default-connection-close.patch added I have chased this down to an opportunistic setting of 'Connection: close' header if-and-only-if #request is called when #started? is false. Attached is a patchfile where I remove this header setting, with the rationale laid out in the description. Hopefully you agree :-) ---------------------------------------- Bug #16559: Net::HTTP#request does not properly close TCP socket if #started? is false https://bugs.ruby-lang.org/issues/16559#change-84047 * Author: f3ndot (Justin Bull) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: 2.8.0-dev, 2.7.0, 2.6.5 * Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN ---------------------------------------- Hello, There appears to be a bug in Net::HTTP#request (and thus #get, #post, etc.) on an instance that isn't explicitly started by the programmer (by invoking #start first, or by executing #request inside a block passed to #start). Inspecting the source code, it reveals #request will recursively call itself inside a #start block if #started? is false. This is great and as I'd expect. However in production and in a test setup I'm observing TCP socket connections on the server-side in the "TIME_WAIT" state, indicating the socket was never properly closed. Conversely, explicitly running #request inside a #start block yields no such behaviour. Consider the following setup, assuming you have docker: ``` docker run --rm -it -p 8080:80/tcp --user root ubuntu apt-get update && apt-get install net-tools watch nginx service nginx start watch 'netstat -tunapl' ``` Running this on your host machine: ``` ruby net = Net::HTTP.new('localhost', 8080) 50.times { net.get('/') } # is bad ``` Will spawn 50 TCP connections on the server, and will all have on TIME_WAIT for 60 seconds (different *nix OSes have different times): ``` Every 2.0s: netstat -tunapl Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 791/nginx: master p tcp 0 0 172.17.0.2:80 172.17.0.1:60772 TIME_WAIT - tcp 0 0 172.17.0.2:80 172.17.0.1:60732 TIME_WAIT - tcp 0 0 172.17.0.2:80 172.17.0.1:60812 TIME_WAIT - tcp 0 0 172.17.0.2:80 172.17.0.1:60778 TIME_WAIT - ... ``` However running any of these incantations have no such result: ``` ruby 50.times { Net::HTTP.get(URI('http://localhost:8080/')) } # is OK ``` ``` ruby net = Net::HTTP.new('localhost', 8080) net.start 50.times { net.get('/') } # is OK net.finish ``` ``` ruby net = Net::HTTP.new('localhost', 8080) 50.times { net.start { net.get('/') } } # is OK ``` These TIME_WAIT connections matter because a server receiving many HTTP requests from clients using Net::HTTP in this fashion (as Faraday does[1]) the server will begin to oversaturate and timeout past a particular scale. I've tested and reproduced this in 2.7 and 2.6. [1]: https://github.com/lostisland/faraday/pull/1117 ---Files-------------------------------- dont-default-connection-close.patch (3.77 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: