From: Brian Candler <B.Candler@...>
Date: 2003-05-16T18:24:38+09:00
Subject: Re: TCP Sockets

On Fri, May 16, 2003 at 05:14:17PM +0900, Dominik Werder wrote:
> I tried to make a simple debugging tool to simulate a simple HTTP request 
> and dump all to my screen. I thought it would be a twoliner but then the 
> blocking issues appeared.. :((

Sending a request is easy enough. Dealing with the response, you can:
- read the headers one line at a time (e.g. using gets)
- end of headers is marked by a blank line
- you then have a Content-Length: header which tells you how many
  bytes to read; or if chunking has been requested, each chunk is preceded
  by a length value. Or for ancient HTTP/0.9 I think you just read until
  the TCP connection closes.

This means you need to understand some of the internal operation of HTTP to
work out how big each read() needs to be. Without using these explicit
mechanisms you can't know exactly when the end of the transaction is:
remember the server response could be split over many TCP segments, and the
segment boundaries are not visible to the application layer anyway. You
could use timeouts (e.g. no data received for 1 second = end of transaction)
but that won't always work because there is no requirement for the HTTP
server to send all or part of its response within a particular time -
consider a slow CGI script which takes 30 seconds to complete, for example.

Regards,

Brian.