classification
Title: Add an option to zipapp to produce a Windows executable
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: paul.moore Nosy List: brett.cannon, cheryl.sabella, donmez, eryksun, miss-islington, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2016-09-22 13:55 by paul.moore, last changed 2018-11-24 09:36 by paul.moore. This issue is now closed.

Files
File name Uploaded Description Edit
zipapp-doc.patch paul.moore, 2016-09-28 11:07 review
Pull Requests
URL Status Linked Edit
PR 6158 merged cheryl.sabella, 2018-03-20 17:23
PR 6163 merged miss-islington, 2018-03-20 22:24
PR 6164 merged miss-islington, 2018-03-20 22:25
Messages (16)
msg277224 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2016-09-22 13:55
The zipapp module allows users to bundle their application as a single file "executable". On Windows, the file is given a ".pyz" extension which is associated with the Python launcher. However, this approach is not always equivalent to a native executable (see http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for more details).

I suggest adding an option to zipapp that prepends a small executable to the zipapp that uses the Python C API to launch the application. A prototype implementation (zastub) is available at https://github.com/pfmoore/pylaunch.

If this seems reasonable, I'll work up a full patch.
msg277228 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-09-22 15:12
Why not just change the extension to cmd and add the following line at the start?

    @python -x "%0" %*
msg277229 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2016-09-22 15:26
(1) It starts an extra process (unless you're running the application from cmd.exe) and (2) in some cases, the system won't recognise a cmd file as an executable. For a simple example,

t.cmd:

@echo Hello from t

example.py:

from subprocess import run
run(["t")]

If you run example.py you get "FileNotFoundError: [WinError 2] The system cannot find the file specified".
msg277231 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-09-22 16:18
Specifically, while CreateProcess does execute batch scripts via the %ComSpec% interpreter, the only extension it infers is ".exe". To run a ".cmd" or ".bat" file, you have to use the full name with the extension.
msg277524 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2016-09-27 16:52
I'm still unsure whether this would be a good idea. On the plus side, it provides (in conjunction with the "embedded" distribution) a really good way of producing a standalone application on Windows. On the minus side there are some limitations which may trip up naive users:

* You can't put C extensions in a zipapp
* You have to take care with things like multiprocessing if you're using an embedded app.

And the wrapper's basically only 4 lines of code:

    wchar_t **myargv = _alloca((__argc + 2) * sizeof(wchar_t*));
    myargv[0] = __wargv[0];
    memcpy(myargv + 1, __wargv, (__argc + 1) * sizeof(wchar_t *));
    return Py_Main(__argc+1, myargv);

so it's not exactly rocket science. (By the way, the arguments to Py_Main are "exactly as those which are passed to a C program’s main" - IIRC, for a C program there's always a NULL pointer at the end of argv, does that rule apply to Py_Main, too? The code doesn't seem to rely on it, so I guess I could save a slot in the array above).

Maybe adding a section to the zipapp docs explaining how to make standalone applications would be a better way of handling this? That would have the advantage of being just as applicable to 3.6 (and 3.5, for that matter) at the cost of making users build their own wrapper (or find a published one).
msg277525 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-09-27 17:10
I think documentation is definitely the cheapest option. It's also the most flexible since if we find out there's a lot of usage of the guide then we can add explicit support.
msg277606 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2016-09-28 11:07
OK, here's a first draft documentation patch. As it's purely a documentation change, I guess it should go into the 3.5, 3.6 and trunk branches? For now it's against trunk (it's not like that file has changed recently anyway), and I'll sort out the merge dance once it looks OK.
msg277638 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-09-28 15:32
Only backport if you want; it's not a bug fix so technically it doesn't need to be backported.
msg314135 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-03-20 10:00
Hi Paul,

Were you interested in moving forward with this doc change?
msg314137 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-03-20 10:42
Hi Cheryl,
Looks like I dropped the ball on this one :-( I think it just needs to be applied to the main branch (3.7/3.8) - I don't think it's worth backporting.

I'll try to get to it when I can, but it may not be for a few weeks, as I have other stuff on my plate at the moment. I've not done anything under the new github workflow, so I don't want to rush it and risk making mistakes. I'm happy if someone else wants to merge it in the meantime.
msg314157 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-03-20 17:25
Hi Paul,

Thanks for your response.  I've made the pull request from your patch, so it would great if you could review it.  Thanks!
msg314171 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-03-20 22:23
New changeset 4be79f29463f632cd8b48486feadc2ed308fb520 by Paul Moore (Cheryl Sabella) in branch 'master':
bpo-28247: Document Windows executable creation in zipapp (GH-6158)
https://github.com/python/cpython/commit/4be79f29463f632cd8b48486feadc2ed308fb520
msg314172 - (view) Author: miss-islington (miss-islington) Date: 2018-03-20 22:44
New changeset a70b8f593657f563116001f654587620271eb9ae by Miss Islington (bot) in branch '3.7':
bpo-28247: Document Windows executable creation in zipapp (GH-6158)
https://github.com/python/cpython/commit/a70b8f593657f563116001f654587620271eb9ae
msg314173 - (view) Author: miss-islington (miss-islington) Date: 2018-03-20 22:52
New changeset 47a0e64ccf711e8d6d0c497d565ca7e2e82de7d4 by Miss Islington (bot) in branch '3.6':
bpo-28247: Document Windows executable creation in zipapp (GH-6158)
https://github.com/python/cpython/commit/47a0e64ccf711e8d6d0c497d565ca7e2e82de7d4
msg330366 - (view) Author: Ismail Donmez (donmez) * Date: 2018-11-23 23:14
The documentation helped a lot, so thanks for that! But it misses the final crucial step:

copy /b zastub.exe+app.pyz app.exe

The documentation talks about prepending the zastub.exe to the zip file but never mentions how, which is very confusing.
msg330374 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-11-24 09:36
While I can see your point, I'm a little skeptical that anyone who's able to write C source code and compile it can't work out how to combine two binary files... :-)
History
Date User Action Args
2018-11-24 09:36:10paul.mooresetmessages: + msg330374
2018-11-23 23:14:45donmezsetnosy: + donmez
messages: + msg330366
2018-03-21 12:43:27cheryl.sabellasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-03-20 22:52:43miss-islingtonsetmessages: + msg314173
2018-03-20 22:44:23miss-islingtonsetnosy: + miss-islington
messages: + msg314172
2018-03-20 22:25:30miss-islingtonsetpull_requests: + pull_request5920
2018-03-20 22:24:40miss-islingtonsetpull_requests: + pull_request5919
2018-03-20 22:23:21paul.mooresetmessages: + msg314171
2018-03-20 17:25:24cheryl.sabellasetmessages: + msg314157
2018-03-20 17:23:46cheryl.sabellasetpull_requests: + pull_request5915
2018-03-20 10:42:51paul.mooresetmessages: + msg314137
versions: + Python 3.8, - Python 3.5, Python 3.6
2018-03-20 10:00:28cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg314135
2016-09-28 15:32:48brett.cannonsetmessages: + msg277638
2016-09-28 11:07:42paul.mooresetfiles: + zipapp-doc.patch
versions: + Python 3.5, Python 3.6
messages: + msg277606

components: + Documentation, - Library (Lib), Windows
keywords: + patch
stage: needs patch -> patch review
2016-09-27 17:10:22brett.cannonsetmessages: + msg277525
2016-09-27 16:52:02paul.mooresetmessages: + msg277524
2016-09-22 16:47:10brett.cannonsetnosy: + brett.cannon
2016-09-22 16:18:27eryksunsetnosy: + eryksun
messages: + msg277231
2016-09-22 15:26:28paul.mooresetmessages: + msg277229
2016-09-22 15:12:36serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg277228
2016-09-22 13:55:29paul.moorecreate