Web APIs

I've been working with web programming for a while now, and I've yet to
find a totally satisfactory API. I like using literate URLs, and no
system so far supports that well.

I'd like to open a discussion about what everyone would like to see in a
web API, so that perhaps such a thing may exist.

Let me sum up some existing ways it's done:

  CGI: It's fairly raw, an undercooked and certainly not object-oriented
  protocol for seeing what the HTTP server (assumably) got, and a
  limited way to respond. You have to know about the deployment
  environment, or introspect from the provided environment  (which is
  often inconsistent) about how you were called.

  FCGI is CGI hacked to be fast -- replace stdin and stdout with sockets
  so they can be opened more than once, and make the app persistent.
  That's about it.

  WEBrick and other servlet APIs: They are far more object-oriented, and
  support design-by-composition of elements. WEBrick makes assumptions
  that it is the only server process involved, so integrating it with
  another server involves heavyweight proxy rewriting, ala Apache's
  mod_proxy.

  mod_ruby and the like let you see any information you want about the
  request, but has similar shortcomings to the rest, and it's specific
  to Apache.

I'd like to create a solution that works for a lot of applications, and
under a lot of web servers, though that would mean writing API modules
for each server, or translators for some other framework.

Some needed features:

  * URL portability -- if a script lives in a server's namespace at
  /application/feed, and you need to move it to /apllication2/feed, it
  shouldn't have to refer to everything using absolute paths nor
  relative ones when at all possible. It should, at the least, be able
  to ask the API for a URL like:

    #{thisscript}/path
    #{thisserver}/path
    #{thisrequest}/path

  to facilitate more aware applications. In addition, where possible, it
  would be nice to have rewrite support be bidirectional, so if there's
  a rewrite for /foo/bar to /baz/bar, if the app requests a path that
  would ordinarily return /baz/bar, the server could rewrite to /foo/bar
  so the user sees only clean URLs.

  * Registry or inspection of namespace -- I'd love an application to be
  able to find out what other valid paths are within the server, so that
  there's less guesswork in making a non-brittle way of linking
  applications together. Being able to say
  "server.application['wiki'].url" would be really cool, though perhaps
  too simplistic.

  * Object orientation -- Represent all of this in good, clear OO style.
  WEBrick excels here more than the other solutions, but it's still
  pretty tentative in comparison to what it could be.

  * Streamability -- templating has to be optional, and preferably
  separate, so that one could, say, write a streaming video-over-http
  server, or a realtime chat system.

  * Support for more than GET and POST methods. It should be possible to
  write a WebDAV server using the API, without having to modify the
  webserver configuration.

  * Filtering of requests, so I could define a handler to intercept all
  text/http documents served by the server, and apply some
  transformation to them.  This would aid writing applications by
  aggregation rather than making applications monolithic. It brings to
  mind the useful unix concepts of filters and pipes.

There's probably a lot more, but I just want to start the conversation
(and if neccesary, we can form a mailing list just for such a thing) so
we can stop re-inventing the wheel and make something exceptional.

Ari