[ruby-core:24075] [Bug #1696] http downloads are unuseably slow

Issue #1696 has been updated by Steven Hartland.


> But non block is critical for large buffer sizes as select will return the
> socket as ready even if it has only one byte to read.

This makes no difference to the socket level recv, even if the socket isn't
set as none blocking, it will always return what it has. It doesn't wait till
the data size matches size of the buffer passed, so again it depends on how the
lower level methods actually deal with this; which is not clear from the docs.

> I would also recommend the nonblocking call happens before the select. This
> way fast connections will not be taxed by two system calls, while slow
> connections are slow anyway so they can wait for the Errno::EAGAIN exception
> to be handled.

This really should be profiled instead of making assumptions. I say that as I
suspect exception handling is actually quite a bit heavier than a select call,
so if its used in any significant amount, it could cause noticeably higher CPU
load. It could go either way, but its always better to know and document why a
design decision was made than to make just make it on a supposition.

A very quick test of 10 requests for a 157MB from a high speed local connection
shows:
                        user     system      total        real
open.none-blocking      5.195312   4.023438   9.218750 ( 11.924775)
open.blocking-patched   2.312500   4.562500   6.875000 (  9.244287)

N.B. The none-blocking test was done using 1.8 install just replacing protocol.rb
with the nightly snapshot, increasing the buffer to 1MB and removing the rescue
type for compatibility.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1696

----------------------------------------
http://redmine.ruby-lang.org