This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pablogsal
Recipients gaborjbernat, pablogsal, vstinner
Date 2020-01-16.17:35:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1579196103.36.0.1543474467.issue39360@roundup.psfhosted.org>
In-reply-to
Content
This is not a regression: the code only worked on 3.7 because there is a sleep in the pool maintainance loop that makes the pool more likely being more consistent, but is the same problem as there is a race condition. For example, reducing the sleep in the loop (notice 3.8 does not use a sleep but instead uses a more resilient system of sentinels) for the 3.7 version reproduces the problem:

diff --git a/Lib/multiprocessing/pool.py b/Lib/multiprocessing/pool.py
index 3e9a0d6b48..f8d438d87c 100644
--- a/Lib/multiprocessing/pool.py
+++ b/Lib/multiprocessing/pool.py
@@ -410,7 +410,7 @@ class Pool(object):
         # is terminated.
         while thread._state == RUN or (pool._cache and thread._state != TERMINATE):
             pool._maintain_pool()
-            time.sleep(0.1)
+            time.sleep(0.00000001)
         # send sentinel to stop workers
         pool._taskqueue.put(None)
         util.debug('worker handler exiting')

With that patch, 3.7 hangs as well. The problem here is that something regarding the locks inside the SimpleQueue of inbound tasks is not in a consistent state when Python finalization calls __del__.
History
Date User Action Args
2020-01-16 17:35:03pablogsalsetrecipients: + pablogsal, vstinner, gaborjbernat
2020-01-16 17:35:03pablogsalsetmessageid: <1579196103.36.0.1543474467.issue39360@roundup.psfhosted.org>
2020-01-16 17:35:03pablogsallinkissue39360 messages
2020-01-16 17:35:02pablogsalcreate