classification
Title: test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x
Type: Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, vstinner
Priority: normal Keywords: patch

Created on 2017-09-18 16:19 by vstinner, last changed 2017-10-02 15:38 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3857 merged vstinner, 2017-10-02 14:48
Messages (8)
msg302469 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-18 16:19
The process was killed (SIGKILL).

See also bpo-30356.

http://buildbot.python.org/all/builders/x86-64%20Sierra%203.x/builds/765/steps/test/logs/stdio

======================================================================
FAIL: test_many_processes (test.test_multiprocessing_spawn.WithProcessesTestProcess)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/_test_multiprocessing.py", line 505, in test_many_processes
    self.assertEqual(p.exitcode, -signal.SIGTERM)
AssertionError: -9 != -15
msg302537 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-19 16:09
I failed to reproduce the bug on Linux.
msg302680 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-21 10:34
test_many_processes() test was added by Antoine Pitrou in bpo-30589. This test was mentionned in bpo-30703.
msg302681 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-21 10:37
test_many_processes() is made in two steps. The bug occurs at the second step which calls proc.terminate() on processes. Code of the test:

    @classmethod
    def _sleep_some(cls):
        time.sleep(100)

    @classmethod
    def _test_sleep(cls, delay):
        time.sleep(delay)

    def test_many_processes(self):
        if self.TYPE == 'threads':
            self.skipTest('test not appropriate for {}'.format(self.TYPE))

        sm = multiprocessing.get_start_method()
        N = 5 if sm == 'spawn' else 100

        # Try to overwhelm the forkserver loop with events
        procs = [self.Process(target=self._test_sleep, args=(0.01,))
                 for i in range(N)]
        for p in procs:
            p.start()
        for p in procs:
            join_process(p)
        for p in procs:
            self.assertEqual(p.exitcode, 0)

        procs = [self.Process(target=self._sleep_some)
                 for i in range(N)]
        for p in procs:
            p.start()
        time.sleep(0.001)  # let the children start...
        for p in procs:
            p.terminate()
        for p in procs:
            join_process(p)
        if os.name != 'nt':
            for p in procs:
                self.assertEqual(p.exitcode, -signal.SIGTERM) # <--- HERE

I'm not sure about the "time.sleep(0.001)  # let the children start...". It looks like a weak synchronization. Maybe if the system is heavily loaded, the signal is sent before Python registered signal handlers? 1 ms seems short to start Python. On my Linux laptop, it's closer to 15 ms.
msg303535 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-10-02 14:33
> It looks like a weak synchronization.

It is.  I don't remember exactly why I had to add this, I can't reproduce any issue without it anymore...

> the signal is sent before Python registered signal handlers?

Python signal handlers are not relevant here, we're sending SIGTERM which kills the process.
msg303536 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-10-02 14:38
It seems other people have similar issues:
https://github.com/libuv/libuv/issues/1226

Perhaps we need to relax the test on OSX :-/
msg303538 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-02 14:48
Antoine: "It seems other people have similar issues: https://github.com/libuv/libuv/issues/1226 Perhaps we need to relax the test on OSX :-/"

Oh thanks for the confirmation. I proposed a patch to accept -SIGKILL on macOS: PR 3857.
msg303544 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-02 15:27
New changeset e6cfdefa0c2f5bda177e49b228c2c7528f7c239c by Victor Stinner in branch 'master':
bpo-31510: Fix multiprocessing test_many_processes() on macOS (#3857)
https://github.com/python/cpython/commit/e6cfdefa0c2f5bda177e49b228c2c7528f7c239c
History
Date User Action Args
2017-10-02 15:38:08vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-10-02 15:27:37vstinnersetmessages: + msg303544
2017-10-02 14:48:59vstinnersetmessages: + msg303538
2017-10-02 14:48:14vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request3837
2017-10-02 14:38:42pitrousetmessages: + msg303536
2017-10-02 14:33:40pitrousetmessages: + msg303535
2017-09-21 10:37:36vstinnersetmessages: + msg302681
2017-09-21 10:34:51vstinnersetnosy: + pitrou
2017-09-21 10:34:43vstinnersetmessages: + msg302680
2017-09-19 16:09:28vstinnersetmessages: + msg302537
2017-09-18 16:19:32vstinnercreate