From: "naruse (Yui NARUSE)" <naruse@...>
Date: 2012-06-07T04:08:57+09:00
Subject: [ruby-core:45447] [ruby-trunk - Feature #6492] Inflate all HTTP Content-Encoding: deflate, gzip, x-gzip responses by default


Issue #6492 has been updated by naruse (Yui NARUSE).


drbrain (Eric Hodel) wrote:
> I've updated this patch.  Upon working with the code again and looking at RFC 2616, I have made the following changes:
> 
> > naruse (Yui NARUSE) wrote:
> > > If Inflater's @socket.read returns nil or a string shorter than clen, it means the input is finished and @inflate can finish.
> > > So at that time, you can call @inflate.finish.
> >
> > I hadn't thought of that, I will implement it.
> 
> Due to read_chunked, and persistent connections I don't see how to make this work.

Yeah, I thought adding an another layer, transport encoding decoder, but it is just an idea and I don't suggest it.

> > this variable inflater is confusing with the inflater method.
> 
> In Net::HTTPResponse#read_chunked, the confusing "inflater" variable has been replaced with "chunk_data_io" which comes from RFC 2616 section 3.6.1.
> 
> > This read method return a string whose length is not clen, this is wrong.
> > Other IO-like object for example Zlib::GzipReader returns a string whose length is clen.
> > So Inflater should have a internal buffer and return the string whose length is just clen.
> 
> Upon review, I think this is OK.
> 
> RFC 2616 specifies that Content-Length and Content-Range (which are used for clen) refer to the transferred bytes and are used to read the correct amount of data from the response to maintain the persistent connection.  Net::HTTPResponse#read_body doesn't allow the user to specify the amount of bytes they wish to read, so returning more data to the user is OK.

Your patch hides content-encoding layer.
Content-Length and Content-Range are the member of the layer.

Net:HTTPRequest#read is on the layer.
A user of net/http can't know whether a request used content-encoding or not.
On such situation, it can't be a reason why hidden Content-Encoding layer effects the behavior of read method.

> I have made an additional change beyond your review:
> 
> I've added a Net::ReadAdapter to the Inflater to stream of the encoded response body through inflate without buffering it all.  This will reduce memory consumption for large responses.

ok.
----------------------------------------
Feature #6492: Inflate all HTTP Content-Encoding: deflate, gzip, x-gzip responses by default
https://bugs.ruby-lang.org/issues/6492#change-27041

Author: drbrain (Eric Hodel)
Status: Assigned
Priority: Normal
Assignee: naruse (Yui NARUSE)
Category: lib
Target version: 2.0.0


=begin
This patch moves the compression-handling code from Net::HTTP#get to Net::HTTPResponse to allow decompression to occur by default on any response body.  (A future patch will set the Accept-Encoding on all requests that allow response bodies by default.)

Instead of having separate decompression code for deflate and gzip-encoded responses, (({Zlib::Inflate.new(32 + Zlib::MAX_WBITS)})) is used which automatically detects and inflated gzip-wrapped streams which allows for simpler processing of gzip bodies (no need to create a StringIO).
=end



-- 
http://bugs.ruby-lang.org/