Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(29064)

Side by Side Diff: Doc/library/compileall.rst

Issue 16104: Use multiprocessing in compileall script
Patch Set: Created 5 years, 6 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | Lib/compileall.py » ('j') | Lib/compileall.py » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 :mod:`compileall` --- Byte-compile Python libraries 1 :mod:`compileall` --- Byte-compile Python libraries
2 =================================================== 2 ===================================================
3 3
4 .. module:: compileall 4 .. module:: compileall
5 :synopsis: Tools for byte-compiling all Python source files in a directory tr ee. 5 :synopsis: Tools for byte-compiling all Python source files in a directory tr ee.
6 6
7 7
8 This module provides some utility functions to support installing Python 8 This module provides some utility functions to support installing Python
9 libraries. These functions compile Python source files in a directory tree. 9 libraries. These functions compile Python source files in a directory tree.
10 This module can be used to create the cached byte-code files at library 10 This module can be used to create the cached byte-code files at library
(...skipping 46 matching lines...) Expand 10 before | Expand all | Expand 10 after
57 Read the file ``list`` and add each line that it contains to the list of 57 Read the file ``list`` and add each line that it contains to the list of
58 files and directories to compile. If ``list`` is ``-``, read lines from 58 files and directories to compile. If ``list`` is ``-``, read lines from
59 ``stdin``. 59 ``stdin``.
60 60
61 .. cmdoption:: -b 61 .. cmdoption:: -b
62 62
63 Write the byte-code files to their legacy locations and names, which may 63 Write the byte-code files to their legacy locations and names, which may
64 overwrite byte-code files created by another version of Python. The default 64 overwrite byte-code files created by another version of Python. The default
65 is to write files to their :pep:`3147` locations and names, which allows 65 is to write files to their :pep:`3147` locations and names, which allows
66 byte-code files from multiple versions of Python to coexist. 66 byte-code files from multiple versions of Python to coexist.
67
68 .. cmdoption:: -j N
69
70 Use *N* processes to compile the given directory.
71 If ``0`` is used, then the result of :func:`os.cpu_count()`
72 will be used.
67 73
68 .. versionchanged:: 3.2 74 .. versionchanged:: 3.2
69 Added the ``-i``, ``-b`` and ``-h`` options. 75 Added the ``-i``, ``-b`` and ``-h`` options.
70 76
77 .. versionchanged:: 3.5
78 Added the ``-j`` option.
79
71 There is no command-line option to control the optimization level used by the 80 There is no command-line option to control the optimization level used by the
72 :func:`compile` function, because the Python interpreter itself already 81 :func:`compile` function, because the Python interpreter itself already
73 provides the option: :program:`python -O -m compileall`. 82 provides the option: :program:`python -O -m compileall`.
74 83
75 Public functions 84 Public functions
76 ---------------- 85 ----------------
77 86
78 .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, qu iet=False, legacy=False, optimize=-1) 87 .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, qu iet=False, legacy=False, optimize=-1, processes=None)
79 88
80 Recursively descend the directory tree named by *dir*, compiling all :file:`. py` 89 Recursively descend the directory tree named by *dir*, compiling all :file:`. py`
81 files along the way. 90 files along the way.
82 91
83 The *maxlevels* parameter is used to limit the depth of the recursion; it 92 The *maxlevels* parameter is used to limit the depth of the recursion; it
84 defaults to ``10``. 93 defaults to ``10``.
85 94
86 If *ddir* is given, it is prepended to the path to each file being compiled 95 If *ddir* is given, it is prepended to the path to each file being compiled
87 for use in compilation time tracebacks, and is also compiled in to the 96 for use in compilation time tracebacks, and is also compiled in to the
88 byte-code file, where it will be used in tracebacks and other messages in 97 byte-code file, where it will be used in tracebacks and other messages in
(...skipping 11 matching lines...) Expand all
100 occur. 109 occur.
101 110
102 If *legacy* is true, byte-code files are written to their legacy locations 111 If *legacy* is true, byte-code files are written to their legacy locations
103 and names, which may overwrite byte-code files created by another version of 112 and names, which may overwrite byte-code files created by another version of
104 Python. The default is to write files to their :pep:`3147` locations and 113 Python. The default is to write files to their :pep:`3147` locations and
105 names, which allows byte-code files from multiple versions of Python to 114 names, which allows byte-code files from multiple versions of Python to
106 coexist. 115 coexist.
107 116
108 *optimize* specifies the optimization level for the compiler. It is passed t o 117 *optimize* specifies the optimization level for the compiler. It is passed t o
109 the built-in :func:`compile` function. 118 the built-in :func:`compile` function.
119
120 The optional argument *processes* gives the number of workers
121 used to compile files in parallel. If :mod:`multiprocessing` is
122 unavailable, then a :exc:`ValueError` will be raised in case that
123 *processes* is given.
Jim.J.Jewett 2014/04/24 03:32:26 Almost all the arguments are optional, but this is
110 124
111 .. versionchanged:: 3.2 125 .. versionchanged:: 3.2
112 Added the *legacy* and *optimize* parameter. 126 Added the *legacy* and *optimize* parameter.
113 127
128 .. versionchanged:: 3.5
129 Added the *processes* parameter.
130
114 131
115 .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=Fals e, legacy=False, optimize=-1) 132 .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=Fals e, legacy=False, optimize=-1)
116 133
117 Compile the file with path *fullname*. 134 Compile the file with path *fullname*.
118 135
119 If *ddir* is given, it is prepended to the path to the file being compiled 136 If *ddir* is given, it is prepended to the path to the file being compiled
120 for use in compilation time tracebacks, and is also compiled in to the 137 for use in compilation time tracebacks, and is also compiled in to the
121 byte-code file, where it will be used in tracebacks and other messages in 138 byte-code file, where it will be used in tracebacks and other messages in
122 cases where the source file does not exist at the time the byte-code file is 139 cases where the source file does not exist at the time the byte-code file is
123 executed. 140 executed.
(...skipping 38 matching lines...) Expand 10 before | Expand all | Expand 10 after
162 179
163 # Perform same compilation, excluding files in .svn directories. 180 # Perform same compilation, excluding files in .svn directories.
164 import re 181 import re
165 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 182 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
166 183
167 184
168 .. seealso:: 185 .. seealso::
169 186
170 Module :mod:`py_compile` 187 Module :mod:`py_compile`
171 Byte-compile a single source file. 188 Byte-compile a single source file.
OLDNEW
« no previous file with comments | « no previous file | Lib/compileall.py » ('j') | Lib/compileall.py » ('J')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+