Re: Robust HTTP file upload server in Ruby?

Tony Arcieri <[e-mail]> wrote:
> I'm looking for a server-side HTTP file upload solution in Ruby which would
> allow a partially completed uploads to be resumed with an HTTP PUT (+
> Range?) request.
> 
> Does anything like this already exist, or is it something I'd have to build
> myself?

Hi Tony,

You could probably build a simple Rack handler (like below, just
with actual parsing and safety checking) and run it with any Rack server.

While I've never needed Range: support for uploads, Rainbows! (and
Unicorn) is heavily tested with PUT file uploads, and supports rare
things like chunked request bodies and HTTP trailers.

Rainbows! even supports Revactor / Rev :)

---------- config.ru ---------
use Rack::ContentLength
use Rack::ContentType
require 'put_resume'
run PutResume.new # see below
------------------------------

---------- put_resume.rb ---------
# totally untested, but based on working code I have
# There's nothing Rainbows!-specific here, at all
class PutResume
  def call(env)
    # FIXME: path validation here
    File.open(File.basename(env["PATH_INFO"], 'wb')) do |fp|
      if range = env["HTTP_RANGE"]
        # implement parse_range yourself, the spec allows for multiple
        # ranges in a multipart request, but I consider those rare enough
        # to ignore for now...
        start, count = parse_range(range)

        # not doing anything with count in this example, we'll just
        # assume the client won't send more than (count - start) bytes
        # but beware of malicious clients
        fp.sysseek(start)
      end

      input = env["rack.input"]
      if buf = input.read(0x4000)
        begin
          fp.syswrite(buf)
        end while input.read(0x4000, buf)
      end
    end

    [ 201, {}, [ "Created\n" ] ]
  end
end

--- rainbows.conf.rb ---
Rainbows! do
  use :Revactor # or :Rev, or anything else Rainbows! supports

  # client_max_body_size is new in Rainbows! 0.92.0, the default is
  # only 1M (same as nginx, but you can override it to anything you
  # want (in bytes))
  client_max_body_size 100 * 1024 * 1024
end

# optional, but you probably want these in a production environment
stderr_path "/path/to/stderr.log"
stdout_path "/path/to/stdout.log"
pid "/path/to/rainbows.pid"
worker_processes 2 # or however many you want to run
------------------------

# to run it all:
$ rainbows -c rainbows.conf.rb config.ru

-- 
Eric Wong