From: khaines@...
Date: 2007-09-18T03:07:41+09:00
Subject: Re: Camping or Merb

On Tue, 18 Sep 2007, Roger Pack wrote:

> I guess the concern would be that using swiftiply to handle static
> content might be a little slower than nginx.  Any thoughts?
>
> If this were a speed problem, a possible fix might be to add to
> EventMachine a call along the lines of 'here's a file's path, I want you
> to pipe that file to this socket' or something.  Then maybe swiftiply
> could get about as fast as C based.  If it were slower.  Just some
> random thoughts.  Thanks Kirk.

A little slower than nginx can still be very fast.

My design goals are to be in the 8k-9k files per second range on my 
development hardware, for small files, and preferably > 120 megabytes per 
second for large files.

It is already a somewhat optimized path that lets it be that fast.  Part 
of what I have been working on over the last month (ugh), along with HTTP 
caching support, are some changes to further streamline and optimize that 
path.  However, after a point, additional speed is mostly irrelevant 
because one has already exceeded the bandwidth of one's port.  If you have 
a 100mbs port, you're going to stream large files at a small percentage of 
120 megabytes per second, anyway, and even the typical small image files 
that are common on web sites are going to saturate your bandwidth at a 
speed less than 8k/second, so the fact that nginx can do 16k/second isn't 
so meaningful, most of the time.

That said....I _think_ small files handling may bump up into the 11k to 
12k/second range on my hardware.  Large file streaming will remain about 
the same.  I am tinkering with it as I type this, in fact.

I'm also tinkering with a microframework, tentatively labeled Wisteria 
(might be something up at wisteria.swiftcore.org later this week), based 
off a lot of this work.  It's aimed at things which need to be done very 
fast, or which require a bunch of long running connections (file uploads 
and downloads).

At its simplest, an earlier version of it that I was playing with last 
week would do 16k "Hello World" type requests per second.  I think that if 
this pans out (it's rather directly derived from other Swiftiply work), 
it'll probably end up doing around 11 thousand to 12 thousand simple 
requests per second, and should be able to saturate a gigabit port, for 
upload or download handling, on one or maybe two processes.


Kirk Haines