# HG changeset patch # User Paul Moore # Date 1424435497 0 # Fri Feb 20 12:31:37 2015 +0000 # Node ID a7968f462508f4b3286d56cf3379d0679a4abdfd # Parent da4eeba6ba2088b898a6f2b1cad28d989d733db3 Implement PEP 441 - Improving Python Zip Application Support diff -r da4eeba6ba20 -r a7968f462508 Doc/library/zipapp.rst --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Doc/library/zipapp.rst Fri Feb 20 12:31:37 2015 +0000 @@ -0,0 +1,153 @@ +:mod:`zipapp` --- Manage executable python zip archives +======================================================= + +.. module:: zipapp + :synopsis: Manage executable python zip archives + + +.. index:: + single: Executable Zip Files + +.. versionadded:: 3.5 + +**Source code:** :source:`Lib/zipapp.py` + +-------------- + +This module provides tools to manage the creation of zip files containing +Python code, which can be executed directly by the Python interpreter (see +:ref:`using-on-interface-options`). The module provides both a +:ref:`zipapp-command-line-interface` and a :ref:`zipapp-python-api`. + + +Basic Example +------------- + +The following example shows how the :ref:`command-line-interface` +can be used to create an executable archive from a directory containing +Python code. When run, the archive will execute the "main" function from +the module "myapp" in the archive. + +.. code-block:: sh + + $ python -m zipapp myapp -m "myapp:main" + $ python myapp.pyz + + + +.. _zipapp-command-line-interface: + +Command-Line Interface +---------------------- + +When called as a program from the command line, the following form is used:: + + python -m zipapp [options] directory + +This will create an executable zip archive from the contents of DIRECTORY. +The following options are understood: + +.. program:: zipapp + +.. cmdoption:: -o ARCHIVE, --output=ARCHIVE + + Write the output to a file named ARCHIVE. If this option is not specified, + the output filename will be the same as the input DIRECTORY, with the + extension ".pyz" added. + +.. cmdoption:: -p INTERPRETER, --python=INTERPRETER + + Add a ``#!`` line to the archive specifying INTERPRETER as the command + to run. Also, on Unix, make the archive executable. The default is to + write no ``#!`` line, and not make the file executable. + +.. cmdoption:: -m MAINFN, --main=MAINFN + + Write a ``__main__.py`` file to the archive that executes MAINFN. The + MAINFN argument should have the form "pkg.mod:fn", where "pkg.mod" is a + package/module in the archive, and "fn" is a callable in the given + module. The ``__main__.py`` file will execute that callable. + +.. cmdoption:: -h, --help + + print a short usage message and exit + + +.. _zipapp-python-api: + +Python API +---------- + +The module defines three convenience functions: + + +.. function:: pack(target, directory, interpreter=None, main=None) + + Pack the specified *directory* into a zip archive named *target*. If + *interpreter* is given, write a line to the start of the file of the form + ``#!\n``, and make the file executable. This causes the file + to be executed by the specified interpreter (the OS handles this on Unix, + and the :ref:`launcher` does so on Windows). If *main* is given, it should + be a string of the form "package.module:callable", and a ``__main__.py`` + file will be written to the archive which imports and calls the given + callable. + + It is an error to supply the *main* argument if the given *directory* + already contains a file ``__main__.py``. It is also an error to omit + *main* if there is no ``__main__.py``. + + +.. function:: get_interpreter(archive) + + Return the interpreter specified in the ``#!`` line at the start of the + archive. If there is no ``#!`` line, return :const:`None`. + +.. function:: set_interpreter(archive, new_archive, interpreter=None) + + Copy *archive* to *new_archive*, replacing the interpreter in the ``#!`` + line with the specified *interpreter*. If *interpreter* is :const:`None`, + delete the ``#!`` line. + + +.. _zipapp-examples: + +Examples +-------- + +Pack up a directory into an archive, and run it. + +.. code-block:: sh + + $ python -m zipapp myapp + $ python myapp.pyz + + +The same can be done using the :func:`pack` functon:: + + >>> import zipapp + >>> zipapp.pack('myapp.pyz', 'myapp') + +To make the application directly executable on Unix, specify an interpreter +to use. + +.. code-block:: sh + + $ python -m zipapp myapp -p "/usr/bin/env python" + $ ./myapp.pyz + + +Note that if you specify an interpreter and then distribute your application +archive, you need to ensure that the interpreter used is portable. The Python +launcher for Windows supports most common forms of Unix ``#!`` line, but there +are other issues to consider: + +* If you use "/usr/bin/env python" (or other forms of the "python" command, + such as "/usr/bin/python"), you need to consider that your users may have + either Python 2 or Python 3 as their default, and write your code to work + under both versions. +* If you use an explicit version, for example "/usr/bin/env python3" your + application will not work for users who do not have that version. (This + may be what you want if you have not made your code Python 2 compatible). +* There is no way to say "python X.Y or later", so be careful of using an + exact version like "/usr/bin/env python3.4" as you will need to change your + shebang line for users of Python 3.5, for example. diff -r da4eeba6ba20 -r a7968f462508 Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst Fri Feb 20 12:46:11 2015 +0200 +++ b/Doc/whatsnew/3.5.rst Fri Feb 20 12:31:37 2015 +0000 @@ -111,6 +111,24 @@ PEP written by Carl Meyer +PEP 441: Improve Python Zip Application Support +----------------------------------------------- + +:pep:`441` adds a :mod:`zipapp` module for creating "Python Zip Applications", +which are zip files containing Python code, which can be executed directly by +the Python interpreter (see :ref:`using-on-interface-options`). Support for +zip applications was added in Python 2.6, but was not well publicised at the +time. + +The Windows installer was also updated to register the ``.pyz`` extension for +Python Zip Applications. + + +.. seealso:: + + :pep:`441` -- Improve Python Zip Application Support + + PEP 475: Retry system calls failing with EINTR ---------------------------------------------- diff -r da4eeba6ba20 -r a7968f462508 Lib/test/test_zipapp.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/test/test_zipapp.py Fri Feb 20 12:31:37 2015 +0000 @@ -0,0 +1,174 @@ +""" +Test harness for the zipapp module. +""" + +import sys +import stat +import pathlib +import tempfile +from test.support import (run_unittest, rmtree) +import unittest +import zipfile +import zipapp + + +class ZipAppTest(unittest.TestCase): + """Test zipapp module functionality.""" + def setUp(self): + self.tmpdir = pathlib.Path(tempfile.mkdtemp()) + + def tearDown(self): + rmtree(str(self.tmpdir)) + + def test_pack(self): + """ + Test packing a directory + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source)) + self.assertTrue(target.is_file()) + + def test_no_main(self): + """ + Test that packing a directory with no __main__.py fails + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / 'foo.py').touch() + target = self.tmpdir / 'source.pyz' + with self.assertRaises(RuntimeError): + zipapp.pack(str(target), str(source)) + + def test_main_and_main_py(self): + """ + Test that supplying a main argument with __main__.py fails + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + with self.assertRaises(RuntimeError): + zipapp.pack(str(target), str(source), main='pkg.mod:fn') + + def test_main_written(self): + """ + Test that the __main__.py is written correctly + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / 'foo.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), main='pkg.mod:fn') + with zipfile.ZipFile(str(target), 'r') as z: + self.assertIn('__main__.py', z.namelist()) + self.assertIn(b'pkg.mod.fn()', z.read('__main__.py')) + + def test_default_no_shebang(self): + """ + Test that no shebang line is written to the target by default + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source)) + with target.open('rb') as f: + self.assertNotEqual(f.read(2), b'#!') + + def test_custom_interpreter(self): + """ + Test that a shebang line with a custom interpreter is written correctly + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter='python') + with target.open('rb') as f: + self.assertEqual(f.read(2), b'#!') + self.assertEqual(b'python\n', f.readline()) + + def test_read_shebang(self): + """ + Test that we can read the shebang line correctly + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter='python') + self.assertEqual(zipapp.get_interpreter(str(target)), 'python') + + def test_read_missing_shebang(self): + """ + Test that we reading the shebang line of a file without one returns None + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source)) + self.assertEqual(zipapp.get_interpreter(str(target)), None) + + def test_write_shebang(self): + """ + Test that we can change the shebang of a file + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter='python') + new_target = self.tmpdir / 'changed.pyz' + zipapp.set_interpreter(str(target), str(new_target), interpreter='python2.7') + self.assertEqual(zipapp.get_interpreter(str(new_target)), 'python2.7') + + def test_remove_shebang(self): + """ + Test that we can remove the shebang from a file + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter='python') + new_target = self.tmpdir / 'changed.pyz' + zipapp.set_interpreter(str(target), str(new_target), interpreter=None) + self.assertEqual(zipapp.get_interpreter(str(new_target)), None) + + # (Unix only) tests that archives with shebang lines are made executable + @unittest.skipIf(sys.platform == 'win32', + 'Windows does not support an executable bit') + def test_shebang_is_executable(self): + """ + Test that a shebang line with a custom interpreter is written correctly + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter='python') + self.assertNotEqual((target.stat().st_mode & stat.S_IEXEC), 0) + + @unittest.skipIf(sys.platform == 'win32', + 'Windows does not support an executable bit') + def test_no_shebang_is_not_executable(self): + """ + Test that a shebang line with a custom interpreter is written correctly + """ + source = self.tmpdir / 'source' + source.mkdir() + (source / '__main__.py').touch() + target = self.tmpdir / 'source.pyz' + zipapp.pack(str(target), str(source), interpreter=None) + self.assertEqual((target.stat().st_mode & stat.S_IEXEC), 0) + + +def test_main(): + run_unittest(ZipAppTest) + +if __name__ == "__main__": + test_main() diff -r da4eeba6ba20 -r a7968f462508 Lib/zipapp.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/zipapp.py Fri Feb 20 12:31:37 2015 +0000 @@ -0,0 +1,113 @@ +from zipfile import ZipFile +import pathlib +import argparse +import os +import sys +import stat + +# The __main__.py used if the users specifies "-m module:fn". +# Note that this will always be written as UTF-8 (module and +# function names can be non-ASCII in Python 3). +MAIN_TEMPLATE = '''\ +# -*- coding: utf-8 -*- +import {module} +{module}.{fn}() +''' + +# The Windows launcher defaults to UTF-8 when parsing shebang lines if the +# file has no BOM. So use UTF-8 on Windows. +# On Unix, use the filesystem encoding. +if sys.platform.startswith('win'): + shebang_encoding = 'utf-8' +else: + shebang_encoding = sys.getfilesystemencoding() + +class PackError(ValueError): + pass + +def write_file_prefix(f, interpreter): + if interpreter: + shebang = b'#!' + interpreter.encode(shebang_encoding) + b'\n' + f.write(shebang) + f.write(b'# Python application packed with zipapp module\n') + +def pack(target, directory, interpreter=None, main=None): + """Pack a directory into an executable Python archive. + + Create a file named TARGET from the contents of DIRECTORY. + The file is a standard zip file, with a prepended shebang line instructing + the OS (or the Python launcher on Windows) to run the file with the + specified INTERPRETER (which defaults to the python executable found + on $PATH). + + The source directory must contain a file named __main__.py in order to + be executable. If no __main__.py is present, the user can specify a + function to call as the entry point, using the MAIN argument. This + should have the form package.module:function, and will be called with + no arguments. + + It is an error to specify MAIN if __main__.py is present, or to omit + MAIN if there is no __main__.py. + """ + + has_main = os.path.exists(os.path.join(directory, '__main__.py')) + if main and has_main: + raise PackError("Cannot spacify entry point if the source has __main__.py") + if not (main or has_main): + raise PackError("Archive has no entry point") + + with open(target, 'wb') as f: + write_file_prefix(f, interpreter) + with ZipFile(f, 'w') as z: + root = pathlib.Path(directory) + for child in root.rglob('*'): + if child.is_file(): + arcname = str(child.relative_to(root)) + z.write(str(child), arcname) + if main: + module, sep, fn = main.partition(':') + main = MAIN_TEMPLATE.format(module=module, fn=fn) + z.writestr('__main__.py', main.encode('utf-8')) + + if interpreter: + os.chmod(target, os.stat(target).st_mode | stat.S_IEXEC) + +def get_interpreter(archive): + with open(archive, 'rb') as f: + if f.read(2) == b'#!': + return f.readline().strip().decode(shebang_encoding) + +def set_interpreter(archive, new_archive, interpreter=None): + with ZipFile(archive) as src: + with open(new_archive, 'wb') as dst_f: + write_file_prefix(f, interpreter) + with ZipFile(dst_f, 'w') as dst: + for info in src.infolist(): + dst.writestr(info, src.read(info)) + + if interpreter: + os.chmod(new_archive, os.stat(new_archive).st_mode | stat.S_IEXEC) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--output', '-o', default=None, + help="The name of the output archive") + parser.add_argument('--python', '-p', default=None, + help="The name of the Python interpreter to use (default: no shebang line)") + parser.add_argument('--main', '-m', default=None, + help="The main function of the application (default: use an existing __main__.py)") + parser.add_argument('directory', + help="Directory to pack into the archive") + + args = parser.parse_args() + + target = args.output if args.output else args.directory + if not os.path.splitext(target)[1]: + target = target + '.pyz' + + pack(target, args.directory, interpreter=args.python, main=args.main) + + +if __name__ == '__main__': + main() diff -r da4eeba6ba20 -r a7968f462508 Tools/msi/exe/exe_en-US.wxl_template --- a/Tools/msi/exe/exe_en-US.wxl_template Fri Feb 20 12:46:11 2015 +0200 +++ b/Tools/msi/exe/exe_en-US.wxl_template Fri Feb 20 12:31:37 2015 +0000 @@ -5,5 +5,5 @@ Python {{ShortVersion}} ({{Bitness}}) Launches the !(loc.ProductName) interpreter. Add to PATH - Adds the install directory to PATH and .py to PATHEXT. + Adds the install directory to PATH and .py/.pyz to PATHEXT. diff -r da4eeba6ba20 -r a7968f462508 Tools/msi/launcher/launcher_en-US.wxl --- a/Tools/msi/launcher/launcher_en-US.wxl Fri Feb 20 12:46:11 2015 +0200 +++ b/Tools/msi/launcher/launcher_en-US.wxl Fri Feb 20 12:31:37 2015 +0000 @@ -5,4 +5,6 @@ Python File Python File (no console) Compiled Python File + Python Zip Application File + Python Zip Application File (no console) diff -r da4eeba6ba20 -r a7968f462508 Tools/msi/launcher/launcher_reg.wxs --- a/Tools/msi/launcher/launcher_reg.wxs Fri Feb 20 12:46:11 2015 +0200 +++ b/Tools/msi/launcher/launcher_reg.wxs Fri Feb 20 12:31:37 2015 +0000 @@ -26,6 +26,20 @@ + + + + + + + + + + + + + + diff -r da4eeba6ba20 -r a7968f462508 Tools/msi/msi.props --- a/Tools/msi/msi.props Fri Feb 20 12:46:11 2015 +0200 +++ b/Tools/msi/msi.props Fri Feb 20 12:31:37 2015 +0000 @@ -79,10 +79,10 @@ $(DefineConstants);CRTRedist=$(CRTRedist); - $(DefineConstants);TestPrefix=;FileExtension=py; + $(DefineConstants);TestPrefix=;FileExtension=py;ArchiveFileExtension=pyz; - $(DefineConstants);TestPrefix=x;FileExtension=px; + $(DefineConstants);TestPrefix=x;FileExtension=px;ArchiveFileExtension=pxz; $(DefineConstants);Suffix32=-32; @@ -157,4 +157,4 @@ $(DefineConstants);@(_UuidValue,';'); - \ No newline at end of file + diff -r da4eeba6ba20 -r a7968f462508 Tools/msi/path/path.wxs --- a/Tools/msi/path/path.wxs Fri Feb 20 12:46:11 2015 +0200 +++ b/Tools/msi/path/path.wxs Fri Feb 20 12:31:37 2015 +0000 @@ -28,6 +28,7 @@ +