From: Szymon Rozga <szymon.rozga@...>
Date: 2005-11-05T19:17:10+09:00
Subject: zlib limitations?

I have encountered a problem with zlib at work. I have to parse through
huge gzipped text files (500000+ lines). However, it seems that zlib
only reads the first 66000 or so lines.

Initially I used the code:

Zlib::GZipReader.open(file) do |f|
  # process text
  puts f.readlines.size
end

>> 66196

However:

IO.open("zcat #{file}",'r') do |io|
  puts io.readlines.size
end

>> 513222

I went through about 2 hours of confusion and struggling not believing
that I couldn't find what I expected using the code, but grepping
returned results. Then I somehow stumbled upon this unexpected cause of
my problems.
Is this a known issue?

-Szymon