classification
Title: Add -t option (timeout) to Tools/buildbot/test.bat for "AMD64 Windows8.1 Refleaks 3.x" buildbot
Type: Stage: resolved
Components: Tests, Windows Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-06-15 11:01 by haypo, last changed 2017-06-26 23:43 by haypo. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2211 merged haypo, 2017-06-15 11:03
PR 2225 merged haypo, 2017-06-15 21:52
PR 2226 merged haypo, 2017-06-15 21:54
PR 2245 merged haypo, 2017-06-16 12:16
PR 2246 merged haypo, 2017-06-16 12:17
PR 2247 merged haypo, 2017-06-16 12:18
Messages (12)
msg296080 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 11:01
I need this option to increase the timeout (currently of 20 min) per test file on the "AMD64 Windows8.1 Refleaks 3.x" buildbot. We already changed the buildbot timeout (timeout x 10 for refleaks tests):
https://github.com/python/buildmaster-config/commit/77316bfe4a5428650d1b120c32d9d65fa69e9c8f

The test step on this buildbot takes 2h30. On the same buildbot slave server, the "AMD64 Windows8.1 Non-Debug 3.x" buildbot takes less than 20 min for the whole "step" step!

For the "x86 Gentoo Refleaks 3.x" buildbot, the buildbot timeout is 12000 seconds and the regrtest timeout is 11700 seconds. But on Windows, we don't support "regrtest timeout" yet, it's currently hardcoded to 20 min.
msg296081 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 11:04
The goal is to fix this failure:

http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Refleaks%203.x/builds/29/steps/test/logs/stdio

Re-running test 'test_largefile' in verbose mode
(...)
test_lseek (test.test_largefile.CLargeFileTest) ... ok
test_osstat (test.test_largefile.CLargeFileTest) ... ok
test_seek_read (test.test_largefile.CLargeFileTest) ... ok
test_seekable (test.test_largefile.CLargeFileTest) ... ok
test_truncate (test.test_largefile.CLargeFileTest) ... ok
test_lseek (test.test_largefile.PyLargeFileTest) ... Timeout (0:20:00)!
Thread 0x0000032c (most recent call first):
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\_pyio.py", line 1617 in write
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\_pyio.py", line 1220 in _flush_unlocked
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\_pyio.py", line 1213 in flush
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\test_largefile.py", line 37 in setUp
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\case.py", line 601 in run
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\case.py", line 653 in __call__
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 122 in run
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\suite.py", line 84 in __call__
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\unittest\runner.py", line 176 in run
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\support\__init__.py", line 1896 in _run_suite
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\support\__init__.py", line 1936 in run_unittest
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\runtest.py", line 164 in test_runner
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\refleak.py", line 84 in dash_R
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\runtest.py", line 167 in runtest_inner
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\runtest.py", line 129 in runtest
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\main.py", line 262 in rerun_failed_tests
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\main.py", line 506 in _main
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\main.py", line 480 in main
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\libregrtest\main.py", line 544 in main
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\test\__main__.py", line 2 in <module>
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\runpy.py", line 85 in _run_code
  File "D:\buildarea\3.x.ware-win81-release.refleak\build\lib\runpy.py", line 193 in _run_module_as_main
msg296121 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 18:19
New changeset 258bfc462b1e58689b43f662a10e44ece3a10bef by Victor Stinner in branch 'master':
bpo-30673: test.bat: add -t option (timeout) (#2211)
https://github.com/python/cpython/commit/258bfc462b1e58689b43f662a10e44ece3a10bef
msg296130 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 22:14
New changeset e064d4dfeda09fd206653697b70b434e98cc1b57 by Victor Stinner in branch '3.6':
bpo-30673: test.bat: add -t option (timeout) (#2211) (#2225)
https://github.com/python/cpython/commit/e064d4dfeda09fd206653697b70b434e98cc1b57
msg296131 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 22:14
New changeset ab4ea09604ea7754a80e4fc36280f5772bf4a8b2 by Victor Stinner in branch '3.5':
bpo-30673: test.bat: add -t option (timeout) (#2211) (#2226)
https://github.com/python/cpython/commit/ab4ea09604ea7754a80e4fc36280f5772bf4a8b2
msg296133 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 22:19
I added the -t options to 3.5, 3.6 and master (3.7) branches.

I have a small issue: in Python 2.7, regrtest has no --timeout option. In Python 3, the timeout was implemented using faulthandler, module which was added to Python 3.3.

I guess that we can handle 2.7 special case in the buildbot configuration.
msg296134 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-15 22:31
Oh wait, it seems like I misunderstood how Tools/buildbot/test.bat works: unknown parameters as passed as if to regrtest. So it wasn't needed to add a new -t option, using --timeout should just work.

There were two bugs in the buildbot configuration:

* the default timeout was not set if no explicit timeout was configured per buildbot
* the timeout was not set on the class used by the "AMD64 Windows8.1 Refleaks 3.x" buildbot

My following configuration change (already deployed on buildbot) fixes these two issues:
https://github.com/python/buildmaster-config/commit/cb5a90724c6a5e316f3b4f8c862bbda142cafc8d
msg296181 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 12:18
> Oh wait, it seems like I misunderstood how Tools/buildbot/test.bat works: unknown parameters as passed as if to regrtest. So it wasn't needed to add a new -t option, using --timeout should just work.

Right, --timeout already worked... I just checked the buildbots.

So I created 3 PR to revert my change to remove my -t parameter :-)
msg296183 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 12:38
New changeset 085a57a8d63b3c8057b5f793518e3d04179710a1 by Victor Stinner in branch '3.6':
Revert "bpo-30673: test.bat: add -t option (timeout) (#2211) (#2225)" (#2246)
https://github.com/python/cpython/commit/085a57a8d63b3c8057b5f793518e3d04179710a1
msg296184 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 12:38
New changeset 2d98c5354239e600cc4f799feabc9cb44666118e by Victor Stinner in branch 'master':
Revert "bpo-30673: test.bat: add -t option (timeout) (#2211)" (#2245)
https://github.com/python/cpython/commit/2d98c5354239e600cc4f799feabc9cb44666118e
msg296185 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 12:38
New changeset f50a3b1e7a74f3132d6b5e1afc793ace6de7e631 by Victor Stinner in branch '3.5':
Revert "bpo-30673: test.bat: add -t option (timeout) (#2211) (#2226)" (#2247)
https://github.com/python/cpython/commit/f50a3b1e7a74f3132d6b5e1afc793ace6de7e631
msg296975 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-26 23:43
I reverted my Tools/buildbot/test.bat change on all branches.

I checked that --timeout is now properly set on the Gentoo Refleaks and Windows Refleaks buildbots on 3.5, 3.6 and master branches.

So this issue is now fixed!
History
Date User Action Args
2017-06-26 23:43:34hayposetstatus: open -> closed
resolution: fixed
messages: + msg296975

stage: resolved
2017-06-16 12:38:52hayposetmessages: + msg296185
2017-06-16 12:38:36hayposetmessages: + msg296184
2017-06-16 12:38:34hayposetmessages: + msg296183
2017-06-16 12:18:49hayposetmessages: + msg296181
2017-06-16 12:18:02hayposetpull_requests: + pull_request2295
2017-06-16 12:17:11hayposetpull_requests: + pull_request2294
2017-06-16 12:16:26hayposetpull_requests: + pull_request2293
2017-06-15 22:31:10hayposetmessages: + msg296134
2017-06-15 22:19:51hayposetmessages: + msg296133
versions: + Python 3.5, Python 3.6
2017-06-15 22:14:47hayposetmessages: + msg296131
2017-06-15 22:14:37hayposetmessages: + msg296130
2017-06-15 21:54:19hayposetpull_requests: + pull_request2271
2017-06-15 21:52:52hayposetpull_requests: + pull_request2270
2017-06-15 18:19:47hayposetmessages: + msg296121
2017-06-15 11:04:33hayposetmessages: + msg296081
2017-06-15 11:03:05hayposetpull_requests: + pull_request2255
2017-06-15 11:01:54haypocreate