[ruby-talk:12933] Re: SV: Re: Looking for a decent compression scheme

>> The reality is that although many compression algorithms can work on
>> streams, that is, they only have to buffer a small, O(1) part of the file
>> at once. Some others (notably bzip2) want direct access to the whole file,
>> so if you could pipe data to it, it'd keep a O(n) buffer. This should
>> affect a lot the way you design the interface in the end.
>> 
>So maybe we need a distinction between StreamCompressor and
>BlockCompressor then? Is there some common interface they share?

No. The algorithm should know whether it needs the entire file (and read it 
in). The user should not have to know whether he is dealing with a block or
a stream algorithm.

I vote for the simplest possible interface --- a compression function #compress
and a decompression function #decompress. However, for efficiency when dealing
with streaming data, we probably want to support both pull mode --- where the
compressor fetches data from the source, and push mode --- where data is pushed
to the compressor.

PULL MODE

The compressor should support the two class methods #compress and #decompress 
with syntax

	Compressor.compress(source)
	Compressor.compress(source, target)

The source is either a string, or an object that can be read(). If the first
form is used, the result is returned as a string. If the second form is used,
the data is write() to the target.

#decompress is similar.

PUSH MODE

In push mode, the functions are:

	co = Compressor.compress_start(target)

Returns a new Compressor object, which keeps track of an ongoing compression.

	co.feed(data)

Feeds some string data to the compressor object. This function is typically
called many times. Whenever output is ready, it is write() to the target
object.

	co.finish()

Informs the Compressor object that all data has been sent. Any remaining data
to be written, is written to the target object.

#decompress_start is similar

DISCUSSION

Note that the pull mode can be trivially implemented in terms of the push mode.
We may want to do this in a mixin CompressPullFromPush.

I don't think we need a class hierarchy for compression, with some abstract
base class at the top. We just say that a compressor is any module supporting
#compress and #compress_start functions.

// Niklas