[ruby-core:63079] [ruby-trunk - Bug #9927] webrick does not unset content-length when responding to HEAD requests

Issue #9927 has been updated by Matthew Kerwin.


Adrien Thebo wrote:
> I didn't make this very clear, but the `curl` invocation hangs for about =
6 seconds until the webrick server is killed, and then prints '`transfer cl=
osed with 23 bytes remaining to read`' which indicates it was hanging.

This is a problem with the way you're using curl. Webrick is doing the righ=
t thing according to the HTTP spec[1], which states that "the payload heade=
r fields [including content-length] MAY be omitted." There is no obligation=
 on the server to do so.

You have to tell your client (curl) not to wait for a message body. -X HEAD=
 just changes bytes omitted in the request, but doesn't change the client b=
ehaviour; you have to use -I to signal that this is a metadata-only fetch [=
2]

[1] http://tools.ietf.org/html/rfc7231#section-4.3.2
[2] http://linux.die.net/man/1/curl

----------------------------------------
Bug #9927: webrick does not unset content-length when responding to HEAD re=
quests
https://bugs.ruby-lang.org/issues/9927#change-47154

* Author: Adrien Thebo
* Status: Rejected
* Priority: Normal
* Assignee:=20
* Category:=20
* Target version:=20
* ruby -v: ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN
----------------------------------------
When Webrick responds to HEAD requests it omits the body (per RFC2616 -- 4.=
4 Message Length). However when setting up the response headers the content=
-length field is set to the length of the body, which means that the result=
ing response will have a content length that doesn't match the actual respo=
nse. This means that some HTTP clients may hang when reading the response.

This is reproducible with the following:

```ruby
require 'webrick'

server =3D WEBrick::HTTPServer.new :Port =3D> 8080, :BindAddress =3D> '127.=
0.0.1'
server.mount_proc("/") do |req, res|
  res.body =3D "This will be ignored!\r\n"
end
trap('INT') do
  server.shutdown
end
server.start
```

Running this with curl results in the following:

~~~
=E2=94=94> ruby webrick-head.rb=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20
[2014-06-10 12:07:28] INFO  WEBrick 1.3.1
[2014-06-10 12:07:28] INFO  ruby 1.9.3 (2013-11-22) [x86_64-linux]
[2014-06-10 12:07:28] INFO  WEBrick::HTTPServer#start: pid=3D24798 port=3D8=
080
localhost - - [10/Jun/2014:12:07:30 PDT] "HEAD / HTTP/1.1" 200 0
- -> /
^C[2014-06-10 12:07:36] INFO  going to shutdown ...
[2014-06-10 12:07:36] INFO  WEBrick::HTTPServer#start done.
~~~

~~~
=E2=94=94> curl -v -X HEAD http://localhost:8080
* Rebuilt URL to: http://localhost:8080/
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Cur=
rent
                                 Dload  Upload   Total   Spent    Left  Spe=
ed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--   =
  0* Connected to localhost (127.0.0.1) port 8080 (#0)
> HEAD / HTTP/1.1
> User-Agent: curl/7.37.0
> Host: localhost:8080
> Accept: */*
>=20
< HTTP/1.1 200 OK=20
* Server WEBrick/1.3.1 (Ruby/1.9.3/2013-11-22) is not blacklisted
< Server: WEBrick/1.3.1 (Ruby/1.9.3/2013-11-22)
< Date: Tue, 10 Jun 2014 19:07:30 GMT
< Content-Length: 23
< Connection: Keep-Alive
<=20

  0    23    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--   =
  0
  0    23    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--   =
  0
  0    23    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--   =
  0
  0    23    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--   =
  0
  0    23    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--   =
  0
[Ctrl-C sent to server]
  0    23    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--   =
  0{ [data not shown]
* transfer closed with 23 bytes remaining to read

  0    23    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--   =
  0
* Closing connection 0
curl: (18) transfer closed with 23 bytes remaining to read
~~~

This is reasonably straightforward to fix; when the headers are being creat=
ed and the code is checking to see if the body should be ignored for HTTP 2=
04 and 304, we can check to see if we're responding to a HEAD request and b=
ehave accordingly. I've attached patches to this effect

---Files--------------------------------
0001-lib-webrick-httpresponse.rb-unset-content-length-hea.patch (1.54 KB)


--=20
https://bugs.ruby-lang.org/