Message345310
Hi guys.
I'm using multiprocessing module to accelerate my program. and I want to do some cleanup work when sub-process exit.
but I found that sub-process would be terminated when registered finalizer are working.
Is this behavior designed intentionally?
code example:
import multiprocessing as mp
from multiprocessing.util import Finalize
import os
import time
def finalizer():
time.sleep(0.2) # some time consuming work
print('do cleanup work: {}'.format(os.getpid()))
def worker(_):
print('do some work: {}'.format(os.getpid()))
def initializer():
# ref: https://github.com/python/cpython/blob/master/Lib/multiprocessing/util.py#L147
Finalize(None, finalizer, exitpriority=1)
# atexit module don't work along with multiprocessing module
# because sub-process exit via os._exit()
# ref: https://docs.python.org/3/library/atexit.html
# atexit.register(finalizer) # don't work
print('main process ID: {}'.format(os.getpid()))
with mp.Pool(4, initializer=initializer) as executor:
executor.map(worker, range(20))
gist link: https://gist.github.com/MrQianJinSi/2daf5b6a9ef08b00facdfbea5200dd28 |
|
Date |
User |
Action |
Args |
2019-06-12 08:35:31 | mrqianjinsi | set | recipients:
+ mrqianjinsi |
2019-06-12 08:35:31 | mrqianjinsi | set | messageid: <1560328531.7.0.625749907508.issue37242@roundup.psfhosted.org> |
2019-06-12 08:35:31 | mrqianjinsi | link | issue37242 messages |
2019-06-12 08:35:31 | mrqianjinsi | create | |
|