From: "kazuhiko (Kazuhiko Shiozaki)" <kazuhiko@...>
Date: 2013-02-24T07:55:59+09:00
Subject: [ruby-core:52752] [Backport 200 - Backport #7831] Net::HTTP does not allow users to handle Content-Encoding in responses like 1.x


Issue #7831 has been updated by kazuhiko (Kazuhiko Shiozaki).


@hsbt found a serious problem with this change.

require 'net/http'
r = Net::HTTP.start('www.iana.org') {|http|
  response = http.get('/domains/example')
}
p r.header.decode_content
puts r.body

Even this small code will display 'false' for r.header.decode_content and gzip'ed binary for body (if the remote http server supports compression).

This issue happens because :
1) http.rb : get() sets accept-encoding = "gzip;q=1.0,deflate;q=0.6,identity;q=0.3" if not specified
2) then http/generic_request.rb : initialize() tries to set decode_content=true ONLY IF accept-encoding does not exist
thus decode_content remains in false and the body is not deflated.

What is not good is that we have 'setting accept-encoding if not specified' code twice, i.e. http.rb : get() and http/generic_request.rb : initialize(). And the latter code does not take into consideration the former code.

AFAIK, there is no other code that sets accept-encoding. So removing 'setting accept-encoding if not specified' code from http.rb : get() should solve this issue.
----------------------------------------
Backport #7831: Net::HTTP does not allow users to handle Content-Encoding in responses like 1.x
https://bugs.ruby-lang.org/issues/7831#change-36841

Author: drbrain (Eric Hodel)
Status: Closed
Priority: Normal
Assignee: drbrain (Eric Hodel)
Category: 
Target version: 


I added a feature to always add a feature to always add Accept-Encoding to HTTP requests and always decode HTTP responses with Content-Encoding.

On Ruby 1.9 and older you could handle Content-Encoding for yourself.

Now Ruby always handles Content-Encoding for you, but does not give you a good indicator that this has taken place. In mechanize this leads to an incompatibility as the Content-Length header is not updated from the original value.

This also disallows handling of bad server responses that browsers handle.

The attached patch (upcoming) addresses this by only handling Content-Decoding in a response if the user did not override the Accept-Encoding header.

Since this is an incompatibility I would like this fixed for Ruby 2.0.  Sorry I did not catch it sooner, I was too busy with RubyGems and RDoc.


-- 
http://bugs.ruby-lang.org/