classification
Title: [3.7] test_multiprocessing_spawn hangs randomly on AppVeyor
Type: Stage: resolved
Components: Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords:

Created on 2018-10-18 06:47 by vstinner, last changed 2018-12-18 22:51 by vstinner. This issue is now closed.

Messages (5)
msg327936 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-18 06:47
On AppVeyor CI, test_multiprocessing_spawn was running for longer than 50 min, whereas usually it only takes around 2 minutes

https://ci.appveyor.com/project/python/cpython/builds/19594644

...
running: test_multiprocessing_spawn (50 min 33 sec)
running: test_multiprocessing_spawn (51 min 3 sec)

Related PR:

https://github.com/python/cpython/pull/9936
msg327938 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-18 06:51
Question: why faulthandler didn't kill the test after 15 min? .github/appveyor.yml runs the test suite using --timeout=1200. Extract of logs:
---
PCbuild\rt.bat -q -uall -u-cpu -u-largefile -rwW --slowest --timeout=1200 --fail-env-changed -j0
C:\projects\cpython>"C:\projects\cpython\PCbuild\win32\python.exe"  -u -Wd -E -bb -m test  -uall -u-cpu -u-largefile -rwW --slowest --timeout 1200 --fail-env-changed -j0 
== CPython 3.7.1rc2+ (a4ccd9402d:a4ccd9402d, Oct 17 2018, 23:09:59) [MSC v.1915 32 bit (Intel)]
== Windows-10-10.0.14393-SP0 little-endian
== cwd: C:\projects\cpython\build\test_python_4988
== CPU count: 2
== encodings: locale=cp1252, FS=utf-8
Using random seed 7230291
Run tests in parallel using 4 child processes
0:00:00 [  1/416] test_wait4 skipped
test_wait4 skipped -- object <module 'os' from 'C:\\projects\\cpython\\lib\\os.py'> has no attribute 'fork'
0:00:00 [  2/416] test_augassign passed
(...)
---

See also bpo-34714 and "timeout in test_multiprocessing_spawn x86 Windows7 3.x buildbot" and bpo-34513: "test_multiprocessing_spawn fails on x86 Windows7 3.7 buildbot".
msg327958 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-18 10:16
I identified a race condition:

https://bugs.python.org/issue33966
https://github.com/python/cpython/pull/7966
msg327959 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-18 10:17
Extract of bpo-34714:

  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

That's why I'm asking if this issue can be related to bpo-33966.
msg332095 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-18 22:51
I haven't seen this bug for 2 months, I'm unable to reproduce the bug and I don't know how to fix it. I close the bug as out of date. Reopen it if it comes back.
History
Date User Action Args
2018-12-18 22:51:02vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg332095

stage: resolved
2018-10-18 10:17:56vstinnersetmessages: + msg327959
2018-10-18 10:16:20vstinnersetmessages: + msg327958
2018-10-18 06:51:32vstinnersetmessages: + msg327938
2018-10-18 06:47:25vstinnercreate