From: Brian Candler <B.Candler@...>
Date: 2008-12-02T21:18:30+09:00
Subject: [ruby-core:20211] Re: Promising C coding techniques to reduceMRI's memory use


--7JfCtLOvnd9MIVvH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Tue, Dec 02, 2008 at 06:26:06PM +0900, Yukihiro Matsumoto wrote:
>   * I just checked in to 1.8 head.
>   * next 1.8.7 maintenance release or 1.8.8 will not have the problem.

Many thanks - I hadn't noticed that you had applied the patch to 1.9
already.

While we're at it, I also analysed some issues with WEBrick: is there any
interest in these?

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18454
  -- possible patch in [18565]
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18459
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18460

I included some suggested patches in those posts, but I really wanted some
discussion/feedback on what was the best way forward.

For now I am using a local monkey-patch (attached) which addresses these
issues.

This patch also adds the ability to return a proc as the body of a
HTTPResponse; the proc is passed an output object, and everything written to
it is turned into a HTTP chunk. This is an expansion of the patch in
[18460]. It also increases block size from 4K to 16K.

I could rewrite these changes as an actual patch to WEBrick if there is
interest in applying them, and agreement on the solutions I've used.

Regards,

Brian.

--7JfCtLOvnd9MIVvH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="webrick-patches.rb"

# Copyright (C) 2008 Brian Candler, released under Ruby Licence.
#
# A collection of small monkey-patches to webrick.

require 'webrick'

module WEBrick

  class HTTPRequest
    # Generate HTTP/1.1 100 continue response. See
    # http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18459
    def continue
      if self['expect'] == '100-continue' && @config[:HTTPVersion] >= "1.1"
        @socket.write "HTTP/#{@config[:HTTPVersion]} 100 continue\r\n\r\n"
        @header.delete('expect')
      end
    end
  end

  class HTTPResponse
    alias :orig_setup_header :setup_header
    # Correct termination of streamed HTTP/1.1 responses. See
    # http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18454 and
    # http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18565
    def setup_header
      orig_setup_header
      unless chunked? || @header['content-length']
        @header['connection'] = "close"
        @keep_alive = false
      end
    end

    # Allow streaming of zipfile entry. See
    # http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/18460
    def send_body(socket)
      if @body.respond_to?(:read) then send_body_io(socket)
      elsif @body.respond_to?(:call) then send_body_proc(socket)
      else send_body_string(socket)
      end
    end

    # If the response body is a proc, then we invoke it and pass in
    # an object which supports "write" and "<<" methods. This allows
    # arbitary output streaming.
    def send_body_proc(socket)
      if @request_method == "HEAD"
        # do nothing
      elsif chunked?
        @body.call(ChunkedWrapper.new(socket, self))
        _write_data(socket, "0#{CRLF}#{CRLF}")
      else
        size = @header['content-length'].to_i
        @body.call(socket)   # TODO: StreamWrapper which supports offset, size
        @sent_size = size
      end
    end
          
    class ChunkedWrapper
      def initialize(socket, resp)
        @socket = socket
        @resp = resp
      end
      def write(buf)
        return if buf.empty?
        data = ""
        data << format("%x", buf.size) << CRLF
        data << buf << CRLF
        socket = @socket
        @resp.instance_eval {
          _write_data(socket, data)
          @sent_size += buf.size
        }
      end
      alias :<< :write
    end
  end
end

if RUBY_VERSION < "1.9"
  old_verbose, $VERBOSE = $VERBOSE, nil
  # Increase from default of 4K for efficiency, similar to
  # http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/branches/ruby_1_8/lib/net/protocol.rb?r1=11708&r2=12092
  # In trunk the default is 64K and can be adjusted using :InputBufferSize,
  # :OutputBufferSize
  WEBrick::HTTPRequest::BUFSIZE = 16384
  WEBrick::HTTPResponse::BUFSIZE = 16384
  $VERBOSE = old_verbose
end

--7JfCtLOvnd9MIVvH--