classification
Title: bdist_msi upload fails
Type: behavior Stage:
Components: Distutils, Distutils2, Windows Versions: Python 3.4, Python 3.3, Python 3.2, Python 2.7, 3rd party
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: alexis, brian.curtin, christian.heimes, eric.araujo, loewis, python-dev, schmir, tarek
Priority: normal Keywords: patch

Created on 2012-01-06 13:18 by schmir, last changed 2013-02-19 16:15 by christian.heimes.

Files
File name Uploaded Description Edit
fix-bdist_msi-upload.patch schmir, 2012-01-06 13:18 make python setup.py bdist_msi upload work
fix-bdist_msi-upload-with-test.patch eric.araujo, 2012-02-11 05:57 review
Messages (20)
msg150729 - (view) Author: Ralf Schmitt (schmir) Date: 2012-01-06 13:18
Running setup.py bdist_msi upload fails with something like:

c:\Python27\python.exe setup.py bdist_msi upload
running bdist_msi
running build
running build_ext
installing to build\bdist.win-amd64\msi
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\msi
creating build\bdist.win-amd64\msi\Lib
creating build\bdist.win-amd64\msi\Lib\site-packages
copying build\lib.win-amd64-2.7\greenlet.pyd -> build\bdist.win-amd64\msi\Lib\site-packages
running install_headers
creating build\bdist.win-amd64\msi\Include
creating build\bdist.win-amd64\msi\Include\greenlet
copying greenlet.h -> build\bdist.win-amd64\msi\Include\greenlet
running install_egg_info
Writing build\bdist.win-amd64\msi\Lib\site-packages\greenlet-0.3.3-py2.7.egg-info
creating dist
removing 'build\bdist.win-amd64\msi' (and everything under it)
running upload
Linking c:\Users\ralf\home\greenlet\build\lib.win-amd64-2.7\greenlet.pyd to c:\Users\ralf\home\greenlet\greenlet.pyd
error: greenlet-0.3.3: No such file or directory

The attached patch fixes it.
msg150742 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-06 16:49
Thanks!  Given the fragility of distutils, we try to add a test for every fix we commit, but I don’t remember if we already have a basic HTTP server to test the upload command.  If not, I think I’ll commit this after manual testing (my Windows VM crashes on boot, so it may take some time for me to get to this).
msg152427 - (view) Author: Ralf Schmitt (schmir) Date: 2012-02-01 12:55
It's a bug in bdist_msi not in the upload command. You can check that distribution.dist_files point to valid files after running the command.
msg152900 - (view) Author: Ralf Schmitt (schmir) Date: 2012-02-08 19:29
there's a workaround in http://pypi.python.org/pypi/xdistutils#the-bdist-msi-fixed-command
msg153107 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-11 05:57
I don’t know if the bdist_msi command runs without user interaction.  If so, then the test in the attached patch should pass on Windows.  Can you please test it?
msg154728 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-02 04:43
Ralf, could you test my patch?
msg154913 - (view) Author: Ralf Schmitt (schmir) Date: 2012-03-04 22:16
sorry, I'm currently short on time and don't use windows at the moment.

bdist_msi runs without user interaction. just let the buildbots test it.
msg154953 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-05 15:26
New changeset 3d1362fa07c3 by Éric Araujo in branch '3.2':
Make distutils’ upload command work with bdist_msi products (#13719).
http://hg.python.org/cpython/rev/3d1362fa07c3
msg154955 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-05 15:48
New changeset 7671932c98d3 by Éric Araujo in branch '2.7':
Make distutils’ upload command work with bdist_msi products (#13719).
http://hg.python.org/cpython/rev/7671932c98d3
msg154956 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-05 16:04
New changeset 4cfcda9e80cb by Éric Araujo in branch '3.2':
Fix NameError from #13719 fix
http://hg.python.org/cpython/rev/4cfcda9e80cb

New changeset 449c9fc2fc2d by Éric Araujo in branch 'default':
Make packaging’ upload command work with bdist_msi products (#13719)
http://hg.python.org/cpython/rev/449c9fc2fc2d
msg154960 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-05 16:25
Will watch buildbots tomorrow and close if they are OK.  Will backport to distutils2 later.
msg154974 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-05 20:53
Buildbots fail when calling listdir; will try to fix this or revert tomorrow.  Help would be appreciated.
msg155017 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-06 14:50
I’m working on this.
msg155110 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-07 20:00
New changeset 077b42a54803 by Éric Araujo in branch '2.7':
Backout buggy patch for #13719
http://hg.python.org/cpython/rev/077b42a54803
msg155113 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-07 20:08
New changeset 7e629bacec87 by Éric Araujo in branch '3.2':
Backout buggy patch committed for #13719
http://hg.python.org/cpython/rev/7e629bacec87

New changeset 17106d7d34b4 by Éric Araujo in branch 'default':
Remove buggy change for #13719 in packaging
http://hg.python.org/cpython/rev/17106d7d34b4
msg155116 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-07 20:27
I finally got a virtual machine up and running and was able to diagnose the problem.  There are two things.  First, the dist directory (where the msi file will be created) is created relative to the current working directory, which explains why listdir in the temporary project directory caused the test to fail.  Second, after your patch bdist_wininst puts a full path in the dist.dist_files list, but it should use a relative path, like other bdist commands do.  (This choice comes from the fact that setup scripts must be run from their parent directory.)

Fixing the first problem is just a matter of adding os.chdir(project_dir) in the test; the second problem is also easy, using os.path.join and self.dist_dir.  I chose to backout the commits because I won’t be able to make a correct patch right now, I have to configure file sharing between my host and my VM.  Let’s have the buildbots green for a while and then break them again (not :).

Debugging this made me realize that all tests should use os.path.join('dist', filename) instead of e.g. 'dist/blah-1.0.rpm' in test_bdist_rpm; I’ll do that in another commit.
msg155117 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-03-07 20:30
I forgot to mention two other problems in my test:
- need to pass name and version keyword arguments to self.create_dist or to change the expected filename to UNKNOWN-UNKNOWN
- need to embed the result of distutils.util.get_platform() into the expected filename
msg178780 - (view) Author: Ralf Schmitt (schmir) Date: 2013-01-01 23:05
AFAIK pypi.python.org disallows uploads of .msi files. That means I've lost interest in this issue. 

Feel free to close it.
msg178790 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2013-01-01 23:53
schmir: your information on msi support is probably outdated; pypi.python.org does support MSI files, see for example

http://pypi.python.org/pypi/Twisted/12.3.0
msg182394 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-19 16:15
I have been bitten by the bug today, too.
History
Date User Action Args
2013-02-19 16:15:03christian.heimessetnosy: + christian.heimes

messages: + msg182394
versions: + Python 3.4
2013-01-01 23:53:17loewissetmessages: + msg178790
2013-01-01 23:05:23schmirsetmessages: + msg178780
2012-03-07 20:30:19eric.araujosetmessages: + msg155117
2012-03-07 20:27:52eric.araujosetresolution: fixed ->
messages: + msg155116
stage: resolved ->
2012-03-07 20:08:23python-devsetmessages: + msg155113
2012-03-07 20:00:53python-devsetmessages: + msg155110
2012-03-06 14:50:58eric.araujosetmessages: + msg155017
2012-03-05 20:53:11eric.araujosetstatus: pending -> open

messages: + msg154974
2012-03-05 16:26:00eric.araujosetstatus: open -> pending
resolution: fixed
messages: + msg154960

stage: test needed -> resolved
2012-03-05 16:04:49python-devsetmessages: + msg154956
2012-03-05 15:48:28python-devsetmessages: + msg154955
2012-03-05 15:26:03python-devsetnosy: + python-dev
messages: + msg154953
2012-03-04 22:16:58schmirsetmessages: + msg154913
2012-03-02 04:43:52eric.araujosetmessages: + msg154728
2012-02-11 05:57:27eric.araujosetfiles: + fix-bdist_msi-upload-with-test.patch

messages: + msg153107
2012-02-08 19:29:51schmirsetmessages: + msg152900
2012-02-01 12:55:40schmirsetmessages: + msg152427
2012-01-06 18:40:10brian.curtinsetnosy: + brian.curtin
type: behavior
components: + Windows
2012-01-06 16:49:21eric.araujosetassignee: tarek -> eric.araujo
components: + Distutils2
versions: + 3rd party, Python 3.3, - Python 3.1
nosy: + alexis

messages: + msg150742
stage: test needed
2012-01-06 13:18:38schmircreate