[ruby-core:57423] [ruby-trunk - Bug #8875] Select is not usable with SSLSocket

Issue #8875 has been updated by headius (Charles Nutter).


akr (Akira Tanaka) wrote:
> 2013/9/8 headius (Charles Nutter) <[e-mail]>:
>  
>  > I would agree, except that users are shown, through examples online and in source, that SSLSocket is "IO-like" and can be used anywhere an IO can be used. IO can do buffered IO and still be selectable. SSLSocket cannot.
>  >
>  > This is a shame because without the buffering in buffering.rb, it *would* be feasible to make select work with SSLSocket, as I have done in JRuby.
>  
>  What's happen when the remote side of SSL protocol sends a partial record?
>  In that case, select notify readability (of encrypted data) but no decrypted
>  data readable.

Doing a select followed by a blocking read of more data than is actually available would block on any socket. The amount of data available in this case is therefore 0 even if the socket is readable.

You should use read_nonblock in combination with select, so only what is available without blocking gets read off the wire. In this case, it would be an empty result (empty string or nil) or raise EAGAIN. The read logic, however, would drain the socket of its partial record into the encrypted buffer. Could we not just associate the buffer check the decrypted buffer, and always attempt to drain the encrypted buffer on any read?

1. Code selects on socket. Decrypted buffer is empty and no data is on the wire yet, so it blocks.
2. Partial record is written and select returns.
3. read_nonblock triggers read from the socket into encrypted buffer.
4. Not enough encrypted data is available to decrypt, so the data remains in the encrypted buffer.
5. read_nonblock raises EAGAIN
6. Code selects on socket. Decrypted buffer is still empty and no data is on the wire yet, so it blocks.
7. The rest of the partial record arrives on the wire. Select returns.
8. read_nonblock drains the rest of the data from the wire into encrypted buffer and drains encrypted buffer into decrypted buffer.

At this point, if the read_nonblock requested all of the decrypted data, that buffer would be drained and select would block again. If read_nonblock requested less than the available data, some would remain in the buffer and select would not block.

>  What's also happen when the remote side request renegotiation and local side
>  write system call blocks?
>  It is difficult to say it is not happen.

It may be that you would need to select for both read and write to ensure this case succeeds. I think it would be a rare occurrence, though, since it will almost always be possible to write into the socket's internal buffer the small amount of data needed for renegotiation.
----------------------------------------
Bug #8875: Select is not usable with SSLSocket
https://bugs.ruby-lang.org/issues/8875#change-42030

Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: MartinBosslet (Martin Bosslet)
Category: ext/openssl
Target version: 
ruby -v: all
Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN


Because of the various levels of buffering SSLSocket employs, it is not possible to reliably use IO.select to check when it has data available.

SSLSocket wraps a normal IO that it uses for reading and writing unencrypted data. This IO has its own buffers, at the OS/libc level.

Select normally operates against IO, checking whether data has been buffered or is available on the wire. However, in order to decrypt data on the wire, SSLSocket often needs to read more data than it needs, potentially draining the stream. This is problem #1.

This problem can be mitigated by making IO.select know that it's an SSLSocket and that it may have its own buffers.

However, there's another layer of buffering that happens in openssl/buffering.rb, where read, readpartial, read_nonblock, and methods that call them eventually hit fill_rbuf, which can easily drain both the IO buffers and the SSLSocket buffers into a Ruby-land buffer IO.select does not know about.

An example script is here: https://gist.github.com/headius/6477345

In investigating why this hangs on JRuby (under the original assumption that it was a JRuby issue) I realized that fill_rbuff is reading 16k bytes at a time to try to fill its internal buffer. This effectively drains all data in all buffers visible to IO.select, causing select to hang after the first read.

ruby-head (a few months old), Ruby 1.9.3p253, Ruby 1.8.7p358, JRuby (all versions), and Rubinius (all versions) are affected, because we all share buffering.rb which is where the problem lies.

This may be a known issue, but we continue to get bug reports from Ruby users claiming JRuby is failing to support select + SSLSocket correctly. I'd like to figure out if there's anything we as a community can do to fix this.


-- 
http://bugs.ruby-lang.org/