classification
Title: concurrent futures Executors accept tasks after interpreter shutdown
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mrknmc, pitrou, tomMoral
Priority: normal Keywords: patch

Created on 2018-03-18 16:41 by mrknmc, last changed 2018-04-10 21:36 by mrknmc. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6144 merged python-dev, 2018-03-18 16:42
PR 6445 merged miss-islington, 2018-04-10 17:24
Messages (6)
msg314044 - (view) Author: Mark Nemec (mrknmc) * Date: 2018-03-18 16:41
Currently, one can submit a task to an executor (both ThreadPoolExecutor and ProcessPoolExecutor) during interpreter shutdown. One way to do this is to register function fun with atexit as below.

@atexit.register
def fun():
   pool.submit(print, "apple")

The future is accepted and goes into PENDING state. However, this can cause issues if the _python_exit function (located in concurrent/futures/thread.py and/or concurrent/futures/process.py) executes before function fun.

Function _python_exit will shutdown the running workers in the pool and hence there will be no workers running by the time fun is executed so the future will be left in PENDING state forever.

The solution submitted here is to instead raise a RuntimeException when a task is submitted during interpreter shutdown. This is the same behaviour as when the shutdown method of an executor is called explicitly.
msg314313 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-03-23 16:35
Thanks for spotting this.  I will take a look soon, unless someone beats me to it.
msg315174 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 17:23
New changeset c4b695f85e141f57d22d8edf7bc2c756da136918 by Antoine Pitrou (Mark Nemec) in branch 'master':
bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144)
https://github.com/python/cpython/commit/c4b695f85e141f57d22d8edf7bc2c756da136918
msg315175 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 18:35
New changeset b26265900a18a184997c3c3a1fa6a5bf29703ec9 by Antoine Pitrou (Miss Islington (bot)) in branch '3.7':
bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144) (GH-6445)
https://github.com/python/cpython/commit/b26265900a18a184997c3c3a1fa6a5bf29703ec9
msg315176 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-04-10 18:38
Thanks your contribution Mark!
msg315182 - (view) Author: Mark Nemec (mrknmc) * Date: 2018-04-10 21:36
Happy to contribute! Thanks for the review :)
History
Date User Action Args
2018-04-10 21:36:03mrknmcsetmessages: + msg315182
2018-04-10 18:38:59pitrousetstatus: open -> closed
versions: - Python 3.6
messages: + msg315176

resolution: fixed
stage: patch review -> resolved
2018-04-10 18:35:11pitrousetmessages: + msg315175
2018-04-10 17:24:31miss-islingtonsetpull_requests: + pull_request6140
2018-04-10 17:23:16pitrousetmessages: + msg315174
2018-03-23 16:35:05pitrousetnosy: + pitrou
messages: + msg314313
2018-03-23 16:34:26pitrousetnosy: + tomMoral

versions: + Python 3.7, Python 3.8
2018-03-18 16:55:48mrknmcsettitle: concurrent.futures executors accept tasks after interpreter shutdown -> concurrent futures Executors accept tasks after interpreter shutdown
2018-03-18 16:42:03python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request5901
2018-03-18 16:41:13mrknmccreate