Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(15549)

Side by Side Diff: Doc/library/compileall.rst

Issue 16104: Use multiprocessing in compileall script
Patch Set: Created 5 years, 4 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | Lib/compileall.py » ('j') | Lib/compileall.py » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 :mod:`compileall` --- Byte-compile Python libraries 1 :mod:`compileall` --- Byte-compile Python libraries
2 =================================================== 2 ===================================================
3 3
4 .. module:: compileall 4 .. module:: compileall
5 :synopsis: Tools for byte-compiling all Python source files in a directory tr ee. 5 :synopsis: Tools for byte-compiling all Python source files in a directory tr ee.
6 6
7 7
8 This module provides some utility functions to support installing Python 8 This module provides some utility functions to support installing Python
9 libraries. These functions compile Python source files in a directory tree. 9 libraries. These functions compile Python source files in a directory tree.
10 This module can be used to create the cached byte-code files at library 10 This module can be used to create the cached byte-code files at library
(...skipping 47 matching lines...) Expand 10 before | Expand all | Expand 10 after
58 Read the file ``list`` and add each line that it contains to the list of 58 Read the file ``list`` and add each line that it contains to the list of
59 files and directories to compile. If ``list`` is ``-``, read lines from 59 files and directories to compile. If ``list`` is ``-``, read lines from
60 ``stdin``. 60 ``stdin``.
61 61
62 .. cmdoption:: -b 62 .. cmdoption:: -b
63 63
64 Write the byte-code files to their legacy locations and names, which may 64 Write the byte-code files to their legacy locations and names, which may
65 overwrite byte-code files created by another version of Python. The default 65 overwrite byte-code files created by another version of Python. The default
66 is to write files to their :pep:`3147` locations and names, which allows 66 is to write files to their :pep:`3147` locations and names, which allows
67 byte-code files from multiple versions of Python to coexist. 67 byte-code files from multiple versions of Python to coexist.
68
69 .. cmdoption:: -j N
70
71 Use *N* processes to compile the given directory.
72 If ``0`` is used, then the result of :func:`os.cpu_count()`
73 will be used.
68 74
69 .. versionchanged:: 3.2 75 .. versionchanged:: 3.2
70 Added the ``-i``, ``-b`` and ``-h`` options. 76 Added the ``-i``, ``-b`` and ``-h`` options.
71 77
78 .. versionchanged:: 3.5
79 Added the ``-j`` option.
80
72 There is no command-line option to control the optimization level used by the 81 There is no command-line option to control the optimization level used by the
73 :func:`compile` function, because the Python interpreter itself already 82 :func:`compile` function, because the Python interpreter itself already
74 provides the option: :program:`python -O -m compileall`. 83 provides the option: :program:`python -O -m compileall`.
75 84
76 Public functions 85 Public functions
77 ---------------- 86 ----------------
78 87
79 .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, qu iet=False, legacy=False, optimize=-1) 88 .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, qu iet=False, legacy=False, optimize=-1, processes=None)
Jim.J.Jewett 2014/04/25 22:33:30 Why is the default value None? If this is just to
Claudiu.Popa 2014/04/27 15:01:18 What should the default be for not using multiple
80 89
81 Recursively descend the directory tree named by *dir*, compiling all :file:`. py` 90 Recursively descend the directory tree named by *dir*, compiling all :file:`. py`
82 files along the way. 91 files along the way.
Jim.J.Jewett 2014/04/25 23:42:22 (Symbolic links to directories are ignored.)
Claudiu.Popa 2014/04/27 15:01:18 Not the subject of the patch. :)
83 92
84 The *maxlevels* parameter is used to limit the depth of the recursion; it 93 The *maxlevels* parameter is used to limit the depth of the recursion; it
85 defaults to ``10``. 94 defaults to ``10``.
86 95
87 If *ddir* is given, it is prepended to the path to each file being compiled 96 If *ddir* is given, it is prepended to the path to each file being compiled
88 for use in compilation time tracebacks, and is also compiled in to the 97 for use in compilation time tracebacks, and is also compiled in to the
89 byte-code file, where it will be used in tracebacks and other messages in 98 byte-code file, where it will be used in tracebacks and other messages in
90 cases where the source file does not exist at the time the byte-code file is 99 cases where the source file does not exist at the time the byte-code file is
91 executed. 100 executed.
92 101
93 If *force* is true, modules are re-compiled even if the timestamps are up to 102 If *force* is true, modules are re-compiled even if the timestamps are up to
94 date. 103 date.
95 104
96 If *rx* is given, its search method is called on the complete path to each 105 If *rx* is given, its search method is called on the complete path to each
97 file considered for compilation, and if it returns a true value, the file 106 file considered for compilation, and if it returns a true value, the file
98 is skipped. 107 is skipped.
99 108
100 If *quiet* is true, nothing is printed to the standard output unless errors 109 If *quiet* is true, nothing is printed to the standard output unless errors
101 occur. 110 occur.
102 111
103 If *legacy* is true, byte-code files are written to their legacy locations 112 If *legacy* is true, byte-code files are written to their legacy locations
104 and names, which may overwrite byte-code files created by another version of 113 and names, which may overwrite byte-code files created by another version of
105 Python. The default is to write files to their :pep:`3147` locations and 114 Python. The default is to write files to their :pep:`3147` locations and
106 names, which allows byte-code files from multiple versions of Python to 115 names, which allows byte-code files from multiple versions of Python to
107 coexist. 116 coexist.
108 117
109 *optimize* specifies the optimization level for the compiler. It is passed t o 118 *optimize* specifies the optimization level for the compiler. It is passed t o
110 the built-in :func:`compile` function. 119 the built-in :func:`compile` function.
120
121 The argument *processes* gives the number of workers
122 used to compile files in parallel. If :mod:`multiprocessing` is
123 unavailable, then a :exc:`ValueError` will be raised in case that
124 *processes* is given.
Jim.J.Jewett 2014/04/25 22:33:30 *argument* --> *parameter*, plus a few nits. The
Jim.J.Jewett 2014/04/25 23:42:22 (change this to ImportError, if appropriate)
111 125
112 .. versionchanged:: 3.2 126 .. versionchanged:: 3.2
113 Added the *legacy* and *optimize* parameter. 127 Added the *legacy* and *optimize* parameter.
114 128
129 .. versionchanged:: 3.5
130 Added the *processes* parameter.
131
115 132
116 .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=Fals e, legacy=False, optimize=-1) 133 .. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=Fals e, legacy=False, optimize=-1)
117 134
118 Compile the file with path *fullname*. 135 Compile the file with path *fullname*.
119 136
120 If *ddir* is given, it is prepended to the path to the file being compiled 137 If *ddir* is given, it is prepended to the path to the file being compiled
121 for use in compilation time tracebacks, and is also compiled in to the 138 for use in compilation time tracebacks, and is also compiled in to the
122 byte-code file, where it will be used in tracebacks and other messages in 139 byte-code file, where it will be used in tracebacks and other messages in
123 cases where the source file does not exist at the time the byte-code file is 140 cases where the source file does not exist at the time the byte-code file is
124 executed. 141 executed.
(...skipping 38 matching lines...) Expand 10 before | Expand all | Expand 10 after
163 180
164 # Perform same compilation, excluding files in .svn directories. 181 # Perform same compilation, excluding files in .svn directories.
165 import re 182 import re
166 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 183 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
167 184
168 185
169 .. seealso:: 186 .. seealso::
170 187
171 Module :mod:`py_compile` 188 Module :mod:`py_compile`
172 Byte-compile a single source file. 189 Byte-compile a single source file.
OLDNEW
« no previous file with comments | « no previous file | Lib/compileall.py » ('j') | Lib/compileall.py » ('J')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+