classification
Title: test_multiprocessing_spawn: test_queue_feeder_donot_stop_onexc() failed on x86 Windows7 3.x
Type: Stage: resolved
Components: Tests Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, pablogsal, vstinner
Priority: normal Keywords: patch

Created on 2019-09-18 09:21 by vstinner, last changed 2019-09-24 12:21 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16348 merged vstinner, 2019-09-24 10:24
PR 16350 merged miss-islington, 2019-09-24 10:48
PR 16351 merged miss-islington, 2019-09-24 10:48
Messages (4)
msg352712 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-18 09:21
x86 Windows7 3.x:
https://buildbot.python.org/all/#/builders/58/builds/3015

If I recall correctly, this buildbot is known to be slow.

It sounds like a race condition.

ERROR: test_queue_feeder_donot_stop_onexc (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1132, in test_queue_feeder_donot_stop_onexc
    self.assertTrue(q.get(timeout=1.0))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\queues.py", line 108, in get
    raise Empty
_queue.Empty


Extract of the test:

    def test_queue_feeder_donot_stop_onexc(self):
        # bpo-30414: verify feeder handles exceptions correctly
        if self.TYPE != 'processes':
            self.skipTest('test not appropriate for {}'.format(self.TYPE))

        class NotSerializable(object):
            def __reduce__(self):
                raise AttributeError
        with test.support.captured_stderr():
            q = self.Queue()
            q.put(NotSerializable())
            q.put(True)
            # bpo-30595: use a timeout of 1 second for slow buildbots
            self.assertTrue(q.get(timeout=1.0))   # <===== FAIL HERE =====
            close_queue(q)

        ...


Hum, "bpo-30595: use a timeout of 1 second for slow buildbots" sounds like this test depends on time which is bad for reliable tests. Would it be possible to use a more reliable synchronization primitive?

If no solution is found, the workaround is to use a timeout of 60 seconds instead of just 1 second...
msg353072 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-24 10:47
New changeset 99799c722065d0524f3ab0bc455e1938bb8dc60f by Victor Stinner in branch 'master':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
https://github.com/python/cpython/commit/99799c722065d0524f3ab0bc455e1938bb8dc60f
msg353073 - (view) Author: miss-islington (miss-islington) Date: 2019-09-24 11:06
New changeset c13a4d6a94b5fab52422fcba612b3d1790aa3422 by Miss Islington (bot) in branch '3.7':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
https://github.com/python/cpython/commit/c13a4d6a94b5fab52422fcba612b3d1790aa3422
msg353074 - (view) Author: miss-islington (miss-islington) Date: 2019-09-24 11:07
New changeset fea9ca1b0c6279f483a73f2040b068dd0e0d3acd by Miss Islington (bot) in branch '3.8':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
https://github.com/python/cpython/commit/fea9ca1b0c6279f483a73f2040b068dd0e0d3acd
History
Date User Action Args
2019-09-24 12:21:40vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-09-24 11:07:31miss-islingtonsetmessages: + msg353074
2019-09-24 11:06:18miss-islingtonsetnosy: + miss-islington
messages: + msg353073
2019-09-24 10:48:25miss-islingtonsetpull_requests: + pull_request15929
2019-09-24 10:48:17miss-islingtonsetpull_requests: + pull_request15928
2019-09-24 10:47:52vstinnersetmessages: + msg353072
2019-09-24 10:24:33vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request15925
2019-09-18 09:21:37vstinnercreate