Author kulikjak
Recipients asvetlov, bquinlan, kartiksubbarao, kulikjak, pitrou, yselivanov
Date 2021-04-15.08:25:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1618475130.4.0.00265762802421.issue43498@roundup.psfhosted.org>
In-reply-to
Content
I investigated a little bit more and found out that this happens when `ProcessPoolExecutor::_adjust_process_count()` adds a new process during the iteration.

With the following change, I can reproduce this reliably every time:

--- Python-3.9.1/Lib/concurrent/futures/process.py
+++ Python-3.9.1/Lib/concurrent/futures/process.py
@@ -373,7 +373,14 @@ class _ExecutorManagerThread(threading.T
         assert not self.thread_wakeup._closed
         wakeup_reader = self.thread_wakeup._reader
         readers = [result_reader, wakeup_reader]
-        worker_sentinels = [p.sentinel for p in self.processes.values()]
+        worker_sentinels = []
+        for p in self.processes.values():
+            time.sleep(1)
+            worker_sentinels.append(p.sentinel)
         ready = mp.connection.wait(readers + worker_sentinels)
 
         cause = None

Since `wait_result_broken_or_wakeup()` is called periodically, and there is no issue if processes added during the iteration are omitted (if they were added just after that, they would be omitted anyway), the attached PR shouldn't break anything.
History
Date User Action Args
2021-04-15 08:25:30kulikjaksetrecipients: + kulikjak, bquinlan, pitrou, asvetlov, yselivanov, kartiksubbarao
2021-04-15 08:25:30kulikjaksetmessageid: <1618475130.4.0.00265762802421.issue43498@roundup.psfhosted.org>
2021-04-15 08:25:30kulikjaklinkissue43498 messages
2021-04-15 08:25:30kulikjakcreate