classification
Title: Native (and default) tarfile support for setup.py sdist in distutils on Windows
Type: Stage:
Components: Distutils Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: tarek Nosy List: georg.brandl, michael.foord, pitrou, r.david.murray, rpetrov, tarek, tim.golden, zooko
Priority: normal Keywords:

Created on 2009-06-17 13:33 by michael.foord, last changed 2009-09-07 08:05 by tarek. This issue is now closed.

Messages (18)
msg89454 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-06-17 13:33
setup.py should be able to generate tarfile distributions on Windows
without requiring tar to be on the path.

Ideally "setup.py sdist" should create the same type of archive (zip or
tarball) by default independent of platform.

At the moment it creates a zip by default on Windows and a tarball on
Linux / OS X.

Packages created on Windows also have CRLF line endings in the MANIFEST
file which is a nuisance for non-Windows consumers of distributions.
msg89455 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-06-17 14:01
Now that distutils uses tarfile (see #6048) we can propose a tar archive
on all platform by default without requiring "tar" to be installed.

Althoug, the gz compression requires the zlib module. I need to check
what are the requirements of its installation in the sdtlib.

For the MANIFEST file, do you mean the file generated by MANIFEST.in ?
msg89456 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-06-17 14:20
Given that Windows can't handle tar files by default but all platforms
support zip out of the box wouldn't (unfortunately) zip be a better default?

For the MANIFEST I meant the file called MANIFEST that distutils creates
(and includes in the distribution) when you run "setup.py sdist".
msg89457 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-06-17 14:29
As a developer I prefer .tar.gz, but I don't have a strong opinion on
picking zip or tar, I think tar is fine as long as all installers out
there (easy_install, pip, etc) are  working with the archives on every
platform too.

Someone who wants more can handle tar imho.

i'll sen a mail to distutils-SIG
msg89461 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-06-17 15:27
The point is not for developers who are happy handling .tar.gz but users
of the distribution who don't have a way of handling them.

I prefer .tar.gz myself as well but I don't think the default
distribution format should be one that isn't natively supported by one
of the major platforms that Python runs on.
msg89462 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2009-06-17 15:29
What's superior about .tar.gz? (This is a genuine question).
msg89463 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-06-17 15:36
Better compression.
msg89464 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-06-17 15:46
I strongly favor a common approach instead of doing it differently on
different platforms.  (Aside: check out the gratuitous differences for
names and locations of distutils config files:
http://docs.python.org/install/index.html#location-and-names-of-config-files
.)

I weakly favor tar over zip.  The current release of tar features LZMA
compression, for example:

$ ls -lS
-rw-rw-r-- 1 zooko zooko 8355840 2009-06-17 09:48
allmydata-tahoe-1.4.1-r3916.tar
-rw-rw-r-- 1 zooko zooko 2562835 2009-06-17 09:51
allmydata-tahoe-1.4.1-r3916.zip
-rw-rw-r-- 1 zooko zooko 2383653 2009-06-17 09:48
allmydata-tahoe-1.4.1-r3916.tar.gz
-rw-rw-r-- 1 zooko zooko 2250149 2009-06-17 09:49
allmydata-tahoe-1.4.1-r3916.tar.bz2
-rw-rw-r-- 1 zooko zooko 2223425 2009-06-17 09:49
allmydata-tahoe-1.4.1-r3916.tar.rz
-rw-rw-r-- 1 zooko zooko 1818466 2009-06-17 09:52
allmydata-tahoe-1.4.1-r3916.7z
-rw-rw-r-- 1 zooko zooko 1811698 2009-06-17 09:50
allmydata-tahoe-1.4.1-r3916.tar.xz

Presumably the current tar module in Python doesn't yet support LZMA,
but it is a good possibility for the future.  On the other hand, ZIP is
more widely used, e.g. by setuptools and especially by tools which were
born in Windows world.
msg89659 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-06-24 09:45
As Michael said. As a Linux user I prefer tar.gz (or tar.bz2 or tar.xz),
but distutils should go with zip since it has better support everywhere.

It's true that tar supports lzma (although unfortunately there is still
no lzma support bundled in the stdlib), but for most source
distributions I don't think size is really critical.
msg89677 - (view) Author: Roumen Petrov (rpetrov) * Date: 2009-06-24 20:03
Antoine, you may mix container with compression. tar as file container
is suitable for unix like systems. other container like zip-container is
not well designed for unix-like file systems.

I disagree with request. Package distribution is platform dependent.
msg89678 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2009-06-24 20:21
Antoine, when you say zip has "better support everywhere", what do you
mean?  I don't want to put words in your mouth, but what I think of is
that users maybe want to pack or unpack distributions with separate
tools instead of with the Python tools.  Is that it?
msg89679 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2009-06-24 20:24
Yup, standard install procedure is (and will probably remain for a
while) - unpack and run python setup.py install

Users should be able to unpack on the most common platforms Python
supports without needing additional tools. All major platforms have
native zip support which isn't true of other formats.
msg89680 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-06-24 20:28
I do not believe it is true that zip is supported by all platforms out
of the box.  As far as I know Gentoo, for example, does not install
unzip by default.  For that matter, before windows XP one had to
download a utility to unzip files on windows (and that utility handles
tar files just fine).  Since there are still a few pre-XP windows boxes
out there, it is not even true to say that all Windows machines handle
zip out of the box.

Personally I would very much dislike it if python source distributions
were zipfiles by default.
msg89691 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-06-25 01:02
> other container like zip-container is
> not well designed for unix-like file systems.

Well, please be more specific as to why zip it affects "sdist" in
particular. Never before have I heard anyone claim that zip was
ill-suited for source tarballs.

> I disagree with request. Package distribution is platform dependent.

It is not package distribution. It is package generation. If you use the
same command ("sdist"), it would be nicer if it produced the same result
regardless of whether you are under Unix or Windows.

(platform-dependent packages, by the way, are rpm, deb, msi, etc. ;-))
msg89692 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-06-25 01:04
> Personally I would very much dislike it if python source distributions
> were zipfiles by default.

Why?
msg89693 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-06-25 02:22
Because I'm a unix weenie, and zip files feel like an intrusion from the
Windows world. I expect source tarballs to be, well, tarballs.  I don't
say zip shouldn't be the default, I just noted that I personally would
find that distasteful. ;)
msg89700 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-06-25 07:46
> Because I'm a unix weenie, and zip files feel like an intrusion from the
> Windows world. I expect source tarballs to be, well, tarballs.  I don't
> say zip shouldn't be the default, I just noted that I personally would
> find that distasteful. ;)

Agreed :)
msg92352 - (view) Author: Tarek Ziadé (tarek) * (Python committer) Date: 2009-09-07 08:05
I think there's no consensus at this point in the "best" format for all
platform.

I am closing this issue as "wontfix". Maybe a new format will rule them
all in a few years.
 
Michael, can you create a specific issue for the CRLF problem ? Thanks a
lot.
History
Date User Action Args
2009-09-07 08:05:04tareksetstatus: open -> closed
resolution: wont fix
messages: + msg92352
2009-06-25 07:46:12georg.brandlsetnosy: + georg.brandl
messages: + msg89700
2009-06-25 02:22:37r.david.murraysetmessages: + msg89693
2009-06-25 01:04:05pitrousetmessages: + msg89692
2009-06-25 01:03:00pitrousetmessages: + msg89691
2009-06-24 20:28:12r.david.murraysetnosy: + r.david.murray
messages: + msg89680
2009-06-24 20:24:42michael.foordsetmessages: + msg89679
2009-06-24 20:21:48zookosetmessages: + msg89678
2009-06-24 20:03:30rpetrovsetnosy: + rpetrov
messages: + msg89677
2009-06-24 09:45:46pitrousetnosy: + pitrou
messages: + msg89659
2009-06-17 15:47:00zookosetnosy: + zooko
messages: + msg89464
2009-06-17 15:36:32michael.foordsetmessages: + msg89463
2009-06-17 15:29:20tim.goldensetnosy: + tim.golden
title: Native (and default) tarfile support for setup.py sdist in distutils on Windows -> Native (and default) tarfile support for setup.py sdist in distutils on Windows
messages: + msg89462
2009-06-17 15:27:15michael.foordsetmessages: + msg89461
2009-06-17 14:29:38tareksetmessages: + msg89457
2009-06-17 14:20:04michael.foordsetmessages: + msg89456
2009-06-17 14:01:40tareksetmessages: + msg89455
versions: + Python 2.7, Python 3.2
2009-06-17 13:33:49michael.foordcreate