Re: Zlib::GzipReader and multiple compressed blobs in a single stream

On 2/2/2011 1:37 PM, Jos Backus wrote:
> It does, but it's very slow. Sigh.

While I don't think you'll be able to make it as fast as zcat, given
that zcat is 100% native code, you might be able to take the
implementation of Zlib::ZReader and tweak it to avoid the need to read
only 1 byte at a time from the delegate stream.  Doing so should speed
things up quite a bit.  The existing code really isn't very involved.
Most of the logic you would need to tweak is in the
Zlib::ZReader#unbuffered_read method, which is actually fairly short.

When @inflater reports that it has finished, it looks like you should be
able to get whatever is left in its input buffer using
@inflater.flush_next_in (from Zlib::ZStream).  Then you can initialize a
new Zlib::Inflater instance and pass that remaining data as the first
input buffer to process.  You would repeat this process every time the
inflater reports it has finished until the end of the delegate is
reached and there is no further data returned by flush_next_in.

If I get some time this evening, I'll look into creating a sample
implementation.  No promises though. :-)

-Jeremy