classification
Title: BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Hans Lawrenz, gvanrossum, python-dev, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2016-04-18 14:43 by Hans Lawrenz, last changed 2016-10-21 21:42 by yselivanov. This issue is now closed.

Files
File name Uploaded Description Edit
run_in_executor_max_workers.patch Hans Lawrenz, 2016-04-18 14:43 review
run_in_executor_max_workers_vcheck.patch Hans Lawrenz, 2016-04-18 15:43 Patch with version check review
run_in_executor_max_workers_with_docs.patch Hans Lawrenz, 2016-04-18 19:01 review
Messages (8)
msg263669 - (view) Author: Hans Lawrenz (Hans Lawrenz) * Date: 2016-04-18 14:43
In issue 21527 <http://bugs.python.org/issue21527> the concurrent.futures.ThreadPoolExecutor was changed to have a default value for max_workers. When asyncio.base_events.BaseEventLoop.run_in_executor creates a default ThreadPoolExecutor it specifies a value of 5 for max_workers, presumably because at the time it was written ThreadPoolExecutor didn't have a default for max_workers. This is confusing because on reading the documentation for ThreadPoolExecutor one might assume that the default specified there is what will be used if a default executor isn't supplied via BaseEventLoop.set_default_executor.

I propose that BaseEventLoop.run_in_executor be changed to not supply a default for max_workers. If this isn't acceptable, a note ought to be put in the run_in_executor documentation.
msg263670 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-04-18 15:19
I like this idea for 3.5 and later. I know this is a pain, but I would like the code to have a version check so that the identical code can still be used in Python 3.3 and 3.4. We go through great lengths to keep the asyncio package compatible with those versions so that we can occasionally push a version out to PyPI for 3.3 and 3.4 users. If the versions diverge we would be unable to keep track of other bug fixes.
msg263671 - (view) Author: Hans Lawrenz (Hans Lawrenz) * Date: 2016-04-18 15:43
Thanks, that makes sense. I've attached a patch with a version check.
msg263672 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-04-18 16:38
There should at least be a comment at the definition of _MAX_WORKERS that it's only used on Python 3.4 and before.

Maybe a note in the docs about the default executor would also be useful.

Otherwise this is exactly what I had in mind -- thanks!

PS. Could you sign a contributor form online
https://www.python.org/psf/contrib/contrib-form/
msg263684 - (view) Author: Hans Lawrenz (Hans Lawrenz) * Date: 2016-04-18 19:01
New patch attached. Includes comments and a note in the documentation.

The documentation note is inside a versionchanged:: 3.5 block. Should this be more specific about the version it changed in? It could be confusing for someone using a version of 3.5 that doesn't have the change.

Contributor form is signed.

Thanks!
msg263685 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-04-18 19:03
LGTM!
msg279161 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-21 21:42
New changeset 785597e758a1 by Yury Selivanov in branch '3.5':
Issue #26796: Don't configure the number of workers for default threadpool executor.
https://hg.python.org/cpython/rev/785597e758a1

New changeset 99941cacfc38 by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #26796)
https://hg.python.org/cpython/rev/99941cacfc38

New changeset a475f2e39c6f by Yury Selivanov in branch 'default':
Merge 3.6 (issue #26796)
https://hg.python.org/cpython/rev/a475f2e39c6f
msg279162 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-10-21 21:42
Thank you, Hans!
History
Date User Action Args
2016-10-21 21:42:35yselivanovsetstatus: open -> closed
versions: + Python 3.6, Python 3.7
messages: + msg279162

resolution: fixed
stage: resolved
2016-10-21 21:42:08python-devsetnosy: + python-dev
messages: + msg279161
2016-04-18 19:03:41gvanrossumsetmessages: + msg263685
2016-04-18 19:01:52Hans Lawrenzsetfiles: + run_in_executor_max_workers_with_docs.patch

messages: + msg263684
2016-04-18 16:38:50gvanrossumsetmessages: + msg263672
2016-04-18 15:43:32Hans Lawrenzsetfiles: + run_in_executor_max_workers_vcheck.patch

messages: + msg263671
2016-04-18 15:19:00gvanrossumsetmessages: + msg263670
2016-04-18 14:43:59Hans Lawrenzcreate