Currently creating tar, gztar, bztar source distributions using the
"--formats=tar,gztar,bztar" distutils option requires external utilities
(tar and possibly one of gzip, bzip2, or compress) to be installed on
the system since distutils uses them.
This is not a problem on most UNIX systems which provide these utilities
by default.  On all other systems (e.g. Windows) it is required to
install such utilities separately and have the executables available in
a directory on your PATH in order to have them work.

I discussed this on distutils mailing list [1] and tried to fix this
hassle. 
The patch in attachment permit to use the distutils
"--formats=tar,gztar,bztar" option without need of having tar/gzip/bzip2
utilities installed by using the tarfile module instead of os.spawn().
It also fixes bug #1885 [2].


[1] http://mail.python.org/pipermail/distutils-sig/2008-January/008654.html
[2] http://bugs.python.org/issue1885

On systems where the gzip or bz2 modules aren't installed, this patch
can raise a tarfile.CompressionError, which should be caught and handled
correctly.  The import of tarfile should also be inside the relevant
function (as is done for make_zipfile) instead of at the top level.

Other than those two points, the patch looks great.

Both things fixed. The new patch is in attachment.

Just one thing: I tried a diff between a tar.gz file generated by
tarfile module and another one generated by the UNIX tar utility and it
seems there are some differences.
I don't know if this could represent a problem for distributions
generated by distutils and eventually easy_install.py integration.
Maybe someone who knows the tarfile implementation could clarify this point.

I just did some tests and could not find any major difference. Which are
the differences you found?

BTW, in your patch the "dist" directory is not automatically created.

> I just did some tests and could not find any major difference. 
> Which are the differences you found?

diff utility doesn't provide any output since the compared files are
binary. As far as I can tell the two files have a different size
(tarfile generates a little bigger file).
Moreover, by using 7zip on Windows I notice that the archive generated
by tarfile module looks different. Once I open it I see the following
structure:

dist (directory) -> archive_name.tar -> archive_name (directory) ->
other subfiles

...while the archive generated with the UNIX tar utility looks like this:

archive_name.tar -> archive_name (directory) -> other subfiles

> BTW, in your patch the "dist" directory is not automatically created.

You're right. patch2.zip file in attachment fixes that.

Hm, on my Linux box both files are more or less identical. Sorry, I
cannot reproduce your problem.

I attach an improved patch1886.use.tarfile.module.diff that fixes all
bugs addressed in this issue.

It also removes second call to compression program leaving that
privilege to tar.

>  I notice that the archive generated by tarfile module looks
> different. Once I open it I see the following structure:
>
> dist (directory) -> archive_name.tar -> archive_name (directory)
> -> other subfiles

This bug is illustrated and patched (not released yet as somebody needs
to reopen it) in issue 1885. The extra path component is taken from the
name passed to GzipFile.

Oh, my mistake - the bug with extra dir component in archive is in issue4750

Second shell call to compression program is removed only for the case
when tarfile or any of required compression modules are not available.

about patch1886.use.tarfile.module.diff - the original tar don't support
-j flag.

What are the systems where does this original tar still exist as a
default utility?

If there is no tarfile module on this systems and "tar" version is so
old then you need a more modern system to wrap your releases or more
modern "tar". The ratio of such systems to windows systems without tar
at all is less than 1%, so it doesn't worth the hassle.

But the true reason why I won't revert the part where command line "tar"
is used, to a double call scheme, is that when you do not pipe "tar"
with compression program and use them separately with relative path -
you will get the same weird header in your tar.gz  That's why it is not
uncommon on windows to meet tar.gz python packages with weird paths
inside. Luckily, windows users are infrequently install something from
sources and if they do - the .zip option is often preferred.

Did I misunderstood something about -j -z -Z flags ?
http://src.opensolaris.org /source/xref/onnv/onnv-gate/usr/src/cmd/tar/tar.c

Roumen, could you be more specific about what are you trying to say with
this 200kB piece of code? Was it the intention to post a link to man or
another piece of spec?

distutils in 2.7 and 3.1 use tarfile.