[ruby-core:45521] [ruby-trunk - Feature #6492] Inflate all HTTP Content-Encoding: deflate, gzip, x-gzip responses by default

Issue #6492 has been updated by drbrain (Eric Hodel).


naruse (Yui NARUSE) wrote:
> drbrain (Eric Hodel) wrote:
> > naruse (Yui NARUSE) wrote:
> > > A user of net/http can't know whether a request used content-encoding or not.
> > 
> > I am unsure what you mean by "can't".
> > 
> > Do you mean "a user of net/http must be able to tell content-encoding was present"?
> 
> Yes.
> 
> > When this patch is combined with #6494 they will not be able to know whether a request used content-encoding or not.  I think this is good, the user should not have to worry about how the bytes were transported.  (This behavior matches Net::HTTP#get).
> 
> Yeah, I think it is acceptable.

Ok.

> > If we want the user to be able to handle content-encoding themselves I think adding a Net::HTTP#compression = false (which will disable both #6492 and #6494) would be best.  We can also add an indicator on Net::HTTPResponse that decompression was performed.
> 
> Someone may want such function, but it is not required until someone actually request.

I will make a future patch, I need such a feature to handle broken deflate streams in mechanize.

> > Content-Range with Content-Encoding requires special handling.  The compressed stream may start anywhere in the underlying block.  (For a deflate-based stream the user would need to manually reconstruct the complete response in order to inflate it.)  I think such users should disable compression.
> 
> So on range response with content-encoding, the response won't uncompress the body
> and keep Content-Encoding field, right?
> If so, I agree.

Ok, I will update the patch

> > > On such situation, it can't be a reason why hidden Content-Encoding layer effects the behavior of read method.
> > 
> > I agree that in RFC 2616 that Content-Encoding, Content-Length and Content-Range are all on the same layer, but without an IO-like interface for the Net::HTTPResponse body I don't think a restriction on the behavior of the read method should apply.  Since this API is entirely internal, I think it is OK if a future addition to the API needs to add buffering to be IO-like.
> 
> OK, I agree if the read method has a comment which explain it breaks the manner of IO-like object
> because it is internal API.

Ok, I will update the patch
----------------------------------------
Feature #6492: Inflate all HTTP Content-Encoding: deflate, gzip, x-gzip responses by default
https://bugs.ruby-lang.org/issues/6492#change-27106

Author: drbrain (Eric Hodel)
Status: Assigned
Priority: Normal
Assignee: naruse (Yui NARUSE)
Category: lib
Target version: 2.0.0


=begin
This patch moves the compression-handling code from Net::HTTP#get to Net::HTTPResponse to allow decompression to occur by default on any response body.  (A future patch will set the Accept-Encoding on all requests that allow response bodies by default.)

Instead of having separate decompression code for deflate and gzip-encoded responses, (({Zlib::Inflate.new(32 + Zlib::MAX_WBITS)})) is used which automatically detects and inflated gzip-wrapped streams which allows for simpler processing of gzip bodies (no need to create a StringIO).
=end



-- 
http://bugs.ruby-lang.org/