From: Jos Backus <jos@...>
Date: 2011-02-04T09:12:11+09:00
Subject: Re: Zlib::GzipReader and multiple compressed blobs in a single stream

On Fri, Feb 04, 2011 at 07:38:04AM +0900, Jeremy Bopp wrote:
> That's great!  How does the performance compare to zcat with your data?

Comparable:

% time zcat gz > /dev/null
zcat gz > /dev/null  0.29s user 0.00s system 99% cpu 0.296 total
% time ./gzr > /dev/null  
./gzr > /dev/null  0.31s user 0.07s system 99% cpu 0.383 total
% 

> BTW, this implementation does require that you have enough memory to
> hold all of the gzipped file data at once.  That will be a problem with
> sufficiently large files or constrained resources.
 
Using the file directly should avoid that. Since we have a File, we don't need
the StringIO object:

    require 'stringio'
    require 'zlib'

    def inflate(filename)
      File.open(filename) do |file|
	zio = file
	loop do
	  io = Zlib::GzipReader.new zio
	  puts io.read
	  unused = io.unused
	  io.finish
	  break if unused.nil?
	  zio.pos -= unused.length
	end
      end
    end

    inflate "gz"

Cheers,
Jos

-- 
Jos Backus
jos at catnook.com