From: bwv549 <jtprince@...>
Date: 2008-09-16T07:47:45+09:00
Subject: Re: how to stream or write data into a tar.gz file as if the data 	were from files?

> Do test it though. I tested it streaming large files in (100MB), and

Yes, upon testing I saw that it was creating a bunch of temp files,
too.  It's too bad since the API is so clean!  Perhaps it will be
reimplemented someday...

********************************************************************
**********************  A solution using Minitar *******************

So, I hacked on archive-tar-minitar for a while and came up with a
solution.  Right now I add a class method that fits with the style of
the pack_file method (indeed, pilfers most of its code) and then I can
access it using the slightly lower level interface than 'pack':

require 'archive/tar/minitar'
require 'stringio'

module Archive::Tar::Minitar

  # entry may be a string (the name), or it may be a hash specifying
the
  # following:
  #   :name    (REQUIRED)
  #   :mode    33188 (rw-r--r--) for files, 16877 (rwxr-xr-x) for dirs
  #           (0O100644)                   (0O40755)
  #   :uid    nil
  #   :gid    nil
  #   :mtime  Time.now
  #
  # if data == nil, then this is considered a directory!
  # (use an empty string for a normal empty file)
  # data should be something that can be opened by StringIO
  def self.pack_as_file(entry, data, outputter) #:yields action, name,
stats:
    outputter = outputter.tar if outputter.kind_of?
(Archive::Tar::Minitar::Output)

    stats = {}
    stats[:uid] = nil
    stats[:gid] = nil
    stats[:mtime] = Time.now

    if data.nil?
      # a directory
      stats[:size] = 4096   # is this OK???
      stats[:mode] = 16877  # rwxr-xr-x
    else
      stats[:size] = data.size
      stats[:mode] = 33188  # rw-r--r--
    end

    if entry.kind_of?(Hash)
      name = entry[:name]

      entry.each { |kk, vv| stats[kk] = vv unless vv.nil? }
    else
      name = entry
    end

    if data.nil?  # a directory
      yield :dir, name, stats if block_given?
      outputter.mkdir(name, stats)
    else          # a file
      outputter.add_file_simple(name, stats) do |os|
        stats[:current] = 0
        yield :file_start, name, stats if block_given?
        StringIO.open(data, "rb") do |ff|
          until ff.eof?
            stats[:currinc] = os.write(ff.read(4096))
            stats[:current] += stats[:currinc]
            yield :file_progress, name, stats if block_given?
          end
        end
        yield :file_done, name, stats if block_given?
      end
    end
  end
end

#####################################
# Then to use it to make a .tgz file:
#####################################

require 'zlib'

file_names = ['a_dir/dorky1', 'dorky2', 'an_empty_dir']
file_data_strings = ['my data', 'my data also', nil]

tgz = Zlib::GzipWriter.new(File.open('my_tar.tgz', 'wb'))

Archive::Tar::Minitar::Output.open(tgz) do |outp|
  file_names.zip(file_data_strings) do |name, data|
    Archive::Tar::Minitar.pack_as_file(name, data, outp)
  end
end

***********************************************************

So, not terribly pretty, but not too terrible either.