Message411208
When Python is built and linked with tcmalloc, using ProcessPoolExecutor may deadlock. Here is a reproducible example:
$ cat t.py
from concurrent import futures
import sys
def work(iteration, item):
sys.stdout.write(f'working: iteration={iteration}, item={item}\n')
sys.stdout.flush()
for i in range(0, 10000):
with futures.ProcessPoolExecutor(max_workers=2) as executor:
executor.submit(work, i, 1)
executor.submit(work, i, 2)
$ python t.py
working: iteration=0, item=1
working: iteration=0, item=2
working: iteration=1, item=1
working: iteration=1, item=2
...
working: iteration=3631, item=1
working: iteration=3631, item=2
<hang here>
The child process fails to finish. It's more likely to reproduce when the system is busy.
With some bisect search internally, this commit https://github.com/python/cpython/commit/1ac6e379297cc1cf8acf6c1b011fccc7b3da2cbe "triggered" the deadlock threshold with tcmalloc. |
|
Date |
User |
Action |
Args |
2022-01-21 23:32:53 | yilei | set | recipients:
+ yilei |
2022-01-21 23:32:53 | yilei | set | messageid: <1642807973.18.0.685289247639.issue46464@roundup.psfhosted.org> |
2022-01-21 23:32:53 | yilei | link | issue46464 messages |
2022-01-21 23:32:53 | yilei | create | |
|