From: Brian Candler <B.Candler@...>
Date: 2003-03-10T03:56:11+09:00
Subject: Net::HTTP optimisation (was: Suggestion for setsockopt)


--qMm9M+Fa2AknHoGS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

I was playing with setsockopt because I was seeing about a 0.1 second
latency on HTTP requests.

However it seems that the underlying problem is that the Net::HTTP library
sends its request as multiple writes (one for the GET or POST line, one for
each header line, one for the blank line, and one for any POST content if
included)

Coalescing these into a single write is straightforward, so I'd like to
propose that at least GET requests be fixed as follows:

module Net
  class HTTPGenericRequest
    def request( sock, ver, path )
      req = sprintf('%s %s HTTP/%s', @method, path, ver) + "\r\n"
      canonical_each do |k,v|
        req << (k + ': ' + v + "\r\n")
      end
      req << "\r\n"
      sock.write req
    end
  end  
end  

That still leaves two writes for a POST (one for the headers, one for the
data). My suggested fix is attached; if the POST data is large then it
doesn't concatenate the header and request, to save building a potentially
large intermediate string.

Doing this reduces the number of network packets sent, gets rid of the
latency problem, and doesn't require any non-portable setsockopt tweaking.

Regards,

Brian.

--qMm9M+Fa2AknHoGS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="httpfast.rb"

require 'net/http'

# This is just an optimisation: sending the request in a single write

module Net
  class HTTPGenericRequest
    def send_request_with_body( sock, ver, path, body )
      if block_given? then
        ac = Accumulator.new
        yield ac              # must be yield, DO NOT USE block.call
        data = ac.terminate
      else
        data = body
      end
      @header['content-length'] = data.size.to_s
      @header.delete 'transfer-encoding'

      unless @header['content-type'] then
        $stderr.puts 'Content-Type did not set; using application/x-www-form-urlencoded' if $VERBOSE
        @header['content-type'] = 'application/x-www-form-urlencoded'
      end

      request sock, ver, path, data
    end

    def request( sock, ver, path, rest="" )
      req = sprintf('%s %s HTTP/%s', @method, path, ver) + "\r\n"
      canonical_each do |k,v|
        req << (k + ': ' + v + "\r\n")
      end
      req << "\r\n"
      if rest.size < 1500   # worth trying to put in single TCP segment
        req << rest
        sock.write req
      else
        sock.write req
        sock.write rest
      end
    end
  end  
end  

--qMm9M+Fa2AknHoGS--