Author asak
Recipients
Date 2006-08-20.01:48:37
SpamBayes Score
Marked as misclassified
Message-id
In-reply-to
Content
In mode w|gz tarfile pads the final block with NULs,
until its size reaches the bufsize value passed to
tarfile.open.  This makes gunzip complain about
"invalid compressed data" because of CRC and length errors.

To reproduce it, put this fragment in a file archive.py


import sys
import tarfile

tar = tarfile.open(mode='w|gz', fileobj=sys.stdout)
tar.close()


and then:
$ python2.5 archive.py | gunzip -c

gunzip: stdin: invalid compressed data--crc error

gunzip: stdin: invalid compressed data--length error

Everything works fine with python 2.3.5 and 2.4.1 on
Debian sarge.

The padding is added by the following lines in
_Stream.close:

blocks, remainder = divmod(len(self.buf), self.bufsize)
if remainder > 0:
    self.buf += NUL * (self.bufsize - remainder)

They were added in revision 38581, but I'm not sure why
- at first sight, "Add tarfile open mode r|*" shouldn't
have to change this write path.

Removing them makes gunzip happy again, but I have no
idea if it breaks something else (test_tarfile doesn't
complain).

A similar problem happens if you use mode w|bz2 and
feed the output to bunzip - it complains about
"trailing garbage after EOF ignored".

Problems found while running the test suite from the
Mercurial SCM.
History
Date User Action Args
2007-08-23 14:42:08adminlinkissue1543303 messages
2007-08-23 14:42:08admincreate