Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(249490)

#16104: Use multiprocessing in compileall script

Can't Edit
Can't Publish+Mail
Start Review
Created:
5 years, 8 months ago by dholth
Modified:
4 years, 11 months ago
Reviewers:
merwok, pcmanticore, jimjjewett, berker.peksag, cf.natali, brett
CC:
brett.cannon, eric.araujo, dholth, Claudiu.Popa, devnull_psf.upfronthosting.co.za, Jim.J.Jewett
Visibility:
Public.

Patch Set 1 #

Patch Set 2 #

Total comments: 4

Patch Set 3 #

Total comments: 4

Patch Set 4 #

Patch Set 5 #

Patch Set 6 #

Total comments: 4

Patch Set 7 #

Patch Set 8 #

Patch Set 9 #

Total comments: 28

Patch Set 10 #

Total comments: 16

Patch Set 11 #

Total comments: 14

Patch Set 12 #

Patch Set 13 #

Patch Set 14 #

Patch Set 15 #

Total comments: 2
Unified diffs Side-by-side diffs Delta from patch set Stats Patch
Doc/library/compileall.rst View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3 chunks +17 lines, -2 lines 0 comments Download
Lib/compileall.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 5 chunks +59 lines, -26 lines 2 comments Download
Lib/test/test_compileall.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3 chunks +59 lines, -0 lines 0 comments Download

Messages

Total messages: 16
eric.araujo
http://bugs.python.org/review/16104/diff/11249/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11249/Doc/library/compileall.rst#newcode118 Doc/library/compileall.rst:118: If *processes* is given, :mod:`concurrent.futures` will be used to ...
5 years, 5 months ago #1
eric.araujo
http://bugs.python.org/review/16104/diff/11272/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11272/Doc/library/compileall.rst#newcode68 Doc/library/compileall.rst:68: .. cmdoption:: -j This option takes an argument. http://bugs.python.org/review/16104/diff/11272/Doc/library/compileall.rst#newcode119 ...
5 years, 5 months ago #2
Claudiu.Popa
http://bugs.python.org/review/16104/diff/11272/Lib/compileall.py File Lib/compileall.py (right): http://bugs.python.org/review/16104/diff/11272/Lib/compileall.py#newcode72 Lib/compileall.py:72: results = executor.map(partial(compile_file, Actually, I'm liking this version more, ...
5 years, 5 months ago #3
eric.araujo
http://bugs.python.org/review/16104/diff/11285/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11285/Doc/library/compileall.rst#newcode70 Doc/library/compileall.rst:70: Use multiple *N* processes to compile the given directory. ...
5 years, 5 months ago #4
Jim.J.Jewett
http://bugs.python.org/review/16104/diff/11298/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11298/Doc/library/compileall.rst#newcode123 Doc/library/compileall.rst:123: *processes* is given. Almost all the arguments are optional, ...
5 years, 3 months ago #5
Claudiu.Popa
http://bugs.python.org/review/16104/diff/11298/Lib/compileall.py File Lib/compileall.py (right): http://bugs.python.org/review/16104/diff/11298/Lib/compileall.py#newcode22 Lib/compileall.py:22: _have_multiprocessing = True On 2014/04/24 03:32:26, Jim.J.Jewett wrote: > ...
5 years, 3 months ago #6
Jim.J.Jewett
http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst#newcode88 Doc/library/compileall.rst:88: .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=False, legacy=False, ...
5 years, 3 months ago #7
Jim.J.Jewett
On 2014/04/24 08:05:18, Claudiu.Popa wrote: > http://bugs.python.org/review/16104/diff/11298/Lib/compileall.py > File Lib/compileall.py (right): > http://bugs.python.org/review/16104/diff/11298/Lib/compileall.py#newcode22 > Lib/compileall.py:22: ...
5 years, 3 months ago #8
Jim.J.Jewett
suggested workaround to the conditional import http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst#newcode91 Doc/library/compileall.rst:91: files along the ...
5 years, 3 months ago #9
Claudiu.Popa
http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11702/Doc/library/compileall.rst#newcode88 Doc/library/compileall.rst:88: .. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=False, legacy=False, ...
5 years, 3 months ago #10
berkerpeksag
http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py File Lib/compileall.py (right): http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py#newcode33 Lib/compileall.py:33: print("Can't list {!r}".format(dir)) Should also this needs to wrap ...
5 years, 3 months ago #11
Claudiu.Popa
http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py File Lib/compileall.py (right): http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py#newcode33 Lib/compileall.py:33: print("Can't list {!r}".format(dir)) On 2014/04/27 15:57:05, berkerpeksag wrote: > ...
5 years, 3 months ago #12
berkerpeksag
On 2014/04/27 16:05:13, Claudiu.Popa wrote: > http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py > File Lib/compileall.py (right): > > http://bugs.python.org/review/16104/diff/11723/Lib/compileall.py#newcode33 > ...
5 years, 3 months ago #13
Charles-François Natali
http://bugs.python.org/review/16104/diff/11723/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11723/Doc/library/compileall.rst#newcode71 Doc/library/compileall.rst:71: Use *N* processes to compile the given directory. The ...
5 years, 3 months ago #14
Claudiu.Popa
http://bugs.python.org/review/16104/diff/11723/Doc/library/compileall.rst File Doc/library/compileall.rst (right): http://bugs.python.org/review/16104/diff/11723/Doc/library/compileall.rst#newcode71 Doc/library/compileall.rst:71: Use *N* processes to compile the given directory. On ...
5 years, 3 months ago #15
brett.cannon
4 years, 11 months ago #16
LGTM; I'll apply shortly.

http://bugs.python.org/review/16104/diff/12846/Lib/compileall.py
File Lib/compileall.py (right):

http://bugs.python.org/review/16104/diff/12846/Lib/compileall.py#newcode76
Lib/compileall.py:76: workers = workers or os.cpu_count()
This can be `workers or None` as that will default to os.cpu_count().

http://bugs.python.org/review/16104/diff/12846/Lib/compileall.py#newcode257
Lib/compileall.py:257: args.workers = args.workers or os.cpu_count()
This can be `args.workers or None`.
Sign in to reply to this message.

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+