Author vstinner
Recipients vstinner
Date 2019-10-16.22:46:46
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1571266006.89.0.625453271604.issue38502@roundup.psfhosted.org>
In-reply-to
Content
Problem.

When regrtest is interrupted by CTRL+c and regrtest multiprocessing code (-jN command line option) is used, regrtest randomly fails to stop come TestWorkerProcess threads.

The problem is that only the direct child process is kill by SIGKILL. If the test spawns a grandchild process, this one will not be killed.

Moreover, the grandchild process inherits the child process stdout and stderr. But regrtest mltiprocessing uses pipes for the child process stdout and stderr.

At the end, Popen.communicate() hangs randomly in the main regrtest process (in a TestWorkerProcess thread) until the child process *and* the grandchild process completes. Except that the main regrtest does not directly kill the grandchild process.

=> see bpo-38207


Solution.

I propose to:

(1) use Popen() with start_new_session=True: a worker process calls setsid() to create a new process group.
(2) Don't kill worker processes, but kill the process groups of worker processes.

Attached PR implements this solution.
History
Date User Action Args
2019-10-16 22:46:46vstinnersetrecipients: + vstinner
2019-10-16 22:46:46vstinnersetmessageid: <1571266006.89.0.625453271604.issue38502@roundup.psfhosted.org>
2019-10-16 22:46:46vstinnerlinkissue38502 messages
2019-10-16 22:46:46vstinnercreate