classification
Title: test_timeout() of test_multiprocessing_spawn.WithManagerTestBarrier fails randomly on x86 Windows7 3.x buildbot
Type: Stage: patch review
Components: Tests Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: davin, pablogsal, vstinner, zach.ware
Priority: normal Keywords: buildbot

Created on 2017-05-09 15:52 by vstinner, last changed 2018-06-20 10:30 by vstinner.

Pull Requests
URL Status Linked Edit
PR 1722 open davin, 2017-05-22 17:07
Messages (9)
msg293328 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-09 15:52
http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/588/steps/test/logs/stdio

0:49:08 [225/405/1] test_multiprocessing_spawn failed
Process Process-59:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1385, in _test_timeout_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 1040, in wait
    return self._callmethod('wait', (timeout,))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
threading.BrokenBarrierError
Process Process-60:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1385, in _test_timeout_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 1040, in wait
    return self._callmethod('wait', (timeout,))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
threading.BrokenBarrierError
Process Process-62:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1385, in _test_timeout_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 1040, in wait
    return self._callmethod('wait', (timeout,))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
threading.BrokenBarrierError
Process Process-61:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1385, in _test_timeout_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 1040, in wait
    return self._callmethod('wait', (timeout,))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
threading.BrokenBarrierError
test_answer_challenge_auth_failure (test.test_multiprocessing_spawn.OtherTest) ... ok
test_deliver_challenge_auth_failure (test.test_multiprocessing_spawn.OtherTest) ... ok

(...)

======================================================================
ERROR: test_timeout (test.test_multiprocessing_spawn.WithManagerTestBarrier)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1399, in test_timeout
    self.run_threads(self._test_timeout_f, (self.barrier, results))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1228, in run_threads
    f(*args)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1385, in _test_timeout_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 1040, in wait
    return self._callmethod('wait', (timeout,))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
threading.BrokenBarrierError

----------------------------------------------------------------------
Ran 277 tests in 762.297s

FAILED (errors=1, skipped=24)
test test_multiprocessing_spawn failed
msg293333 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-09 16:03
Other similar errors.

http://buildbot.python.org/all/builders/x86%20Windows7%203.6/builds/339/steps/test/logs/stdio

1:05:52 [382/405/1] test_multiprocessing_spawn failed -- running: test_mmap (556 sec)
Process Process-129:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1317, in _test_reset_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 603, in wait
    self._enter() # Block while the barrier drains.
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 627, in _enter
    raise BrokenBarrierError
threading.BrokenBarrierError
Process Process-130:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1317, in _test_reset_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 603, in wait
    self._enter() # Block while the barrier drains.
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 627, in _enter
    raise BrokenBarrierError
threading.BrokenBarrierError
Process Process-132:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1317, in _test_reset_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 603, in wait
    self._enter() # Block while the barrier drains.
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 627, in _enter
    raise BrokenBarrierError
threading.BrokenBarrierError
Process Process-131:
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 249, in _bootstrap
    self.run()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1179, in task
    self.f(*self.args)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1317, in _test_reset_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 603, in wait
    self._enter() # Block while the barrier drains.
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 627, in _enter
    raise BrokenBarrierError
threading.BrokenBarrierError
test_answer_challenge_auth_failure (test.test_multiprocessing_spawn.OtherTest) ... ok
test_deliver_challenge_auth_failure (test.test_multiprocessing_spawn.OtherTest) ... ok

(...)

test_barrier_10 (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... ok
test_default_timeout (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... ok
test_reset (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... ERROR
test_single_thread (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... ok
test_thousand (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... ok

(...)

======================================================================
ERROR: test_reset (test.test_multiprocessing_spawn.WithProcessesTestBarrier)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1341, in test_reset
    (self.barrier, results1, results2, results3))
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1228, in run_threads
    f(*args)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1317, in _test_reset_f
    i = barrier.wait()
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 612, in wait
    self._wait(timeout)
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\threading.py", line 650, in _wait
    raise BrokenBarrierError
threading.BrokenBarrierError

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 753, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.16138982772827148 not greater than or equal to 0.17

----------------------------------------------------------------------
Ran 277 tests in 579.477s
msg294166 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2017-05-22 18:06
To better accommodate very slow buildbots, a parameter is added in PR-1722 to scale up the timeout durations where they are necessary on a per-machine basis.

Relevant tests have a timeout set to some default number of seconds times a multiplier value.

The multiplier value can be controlled by the environment variable 'CONF_TIMEOUT_MULTIPLIER' which defaults to a multiplier of 1.0 if not set.  On buildbots, this environment variable can be set by defining a parameter by that name in the buildbot configuration file for a machine.  Otherwise, this environment variable can be set in the usual way before running tests on non-buildbot machines.
msg296443 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-20 14:41
http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/770/steps/test/logs/stdio

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithManagerTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 788, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.15492773056030273 not greater than or equal to 0.17

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithThreadsTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 788, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.16765165328979492 not greater than or equal to 0.17
msg296466 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-20 15:55
http://buildbot.python.org/all/builders/x86%20Windows7%203.5/builds/299/steps/test/logs/stdio

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.5.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 753, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.1293318271636963 not greater than or equal to 0.17
msg296529 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-06-21 08:53
http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/774/steps/test/logs/stdio



======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 788, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.15966796875 not greater than or equal to 0.17

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithThreadsTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 788, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.1293318271636963 not greater than or equal to 0.17
msg317947 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-05-28 23:00
Recent failure on x86 Windows7 3.x:

http://buildbot.python.org/all/#/builders/58/builds/914

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithManagerTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1040, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.1373279094696045 not greater than or equal to 0.17
msg319413 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-06-12 23:52
New failure on x86 Windows7 3.x:

http://buildbot.python.org/all/#/builders/58/builds/1000
msg320058 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-20 10:30
http://buildbot.python.org/all/#/builders/90/builds/414

======================================================================
FAIL: test_timeout (test.test_multiprocessing_spawn.WithThreadsTestQueue)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.6.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 883, in test_timeout
    self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.16714811325073242 not greater than or equal to 0.17
History
Date User Action Args
2018-06-20 10:30:58vstinnersetmessages: + msg320058
2018-06-12 23:52:30pablogsalsetnosy: + pablogsal
messages: + msg319413
2018-05-28 23:04:12ned.deilysetversions: + Python 3.8
2018-05-28 23:00:54vstinnersetmessages: + msg317947
2017-06-21 08:53:26vstinnersetmessages: + msg296529
2017-06-20 15:55:20vstinnersetmessages: + msg296466
2017-06-20 14:41:47vstinnersetmessages: + msg296443
2017-05-22 18:10:13davinlinkissue30339 dependencies
2017-05-22 18:06:05davinsetnosy: + davin, zach.ware

messages: + msg294166
stage: patch review
2017-05-22 17:07:08davinsetpull_requests: + pull_request1810
2017-05-09 16:03:38vstinnersetmessages: + msg293333
2017-05-09 15:52:59vstinnercreate