This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x
Type: Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: davin, pitrou, vstinner
Priority: normal Keywords: buildbot

Created on 2017-07-24 09:15 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2862 merged vstinner, 2017-07-25 12:10
PR 7308 merged vstinner, 2018-06-01 10:29
Messages (6)
msg298940 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-24 09:15
Same question as usual: why regrtest doesn't display the "running: test_multiprocessing_spawn (319 sec)" update every 30 seconds? Is regrtest main process also stuck?

http://buildbot.python.org/all/builders/AMD64%20Windows10%203.x/builds/973/steps/test/logs/stdio

...
Run tests in parallel using 2 child processes
...
0:47:28 [403/406] test_buffer passed (119 sec) -- running: test_multiprocessing_spawn (319 sec)
0:47:30 [404/406] test_turtle passed -- running: test_multiprocessing_spawn (321 sec)
0:47:32 [405/406] test_dynamic passed -- running: test_multiprocessing_spawn (323 sec)

command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-x64', '-j2', '--timeout', '900'], attempting to kill
program finished with exit code 1
elapsedTime=4059.302000
msg298941 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-24 09:17
Same bug on AMD64 Windows8 3.x buildbot:

http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/943/steps/test/logs/stdio

Run tests in parallel using 2 child processes
...
1:03:06 [402/406] test_getargs2 passed -- running: test_multiprocessing_spawn (1203 sec)
1:03:09 [403/406] test_threadedtempfile passed -- running: test_multiprocessing_spawn (1205 sec)
1:03:11 [404/406] test_slice passed -- running: test_multiprocessing_spawn (1208 sec)
1:03:19 [405/406] test_tracemalloc passed -- running: test_multiprocessing_spawn (1215 sec)

command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-x64', '-j2', '--timeout', '900'], attempting to kill
program finished with exit code 1
elapsedTime=5005.881000
msg299050 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-25 11:42
test_multiprocessing_spawn hangs on running test_child_fd_inflation(). Child processes call test.support.fd_count(). But in a debug build, if the fd is invalid, os.dup(fd) kills immediately the process without letting Python to handle the error. It seems like os.dup(fd) raises a OSError(EBADF), as expected, in release mode.

The test is new and currently only in master, see bpo-26732. The commit:

commit 896145d9d266ee2758cfcd7691238cbc1f9e1ab8
Author: Antoine Pitrou <pitrou@free.fr>
Date:   Sat Jul 22 13:22:54 2017 +0200

    bpo-26732: fix too many fds in processes started with the "forkserver" method (#2813)
    
    * bpo-26732: fix too many fds in processes started with the "forkserver" method
    
    A child process would inherit as many fds as the number of still-running children.
    
    * Add blurb and test comment
msg299115 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-25 15:45
New changeset 302bbbe9ba5c72559913e2ea006c921f698a729d by Victor Stinner in branch 'master':
bpo-31009: Fix support.fd_count() on Windows (#2862)
https://github.com/python/cpython/commit/302bbbe9ba5c72559913e2ea006c921f698a729d
msg299173 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-26 00:10
Windows buildbots are green again, I close the issue.
msg318401 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-01 11:47
New changeset 99ba73d1188f4d52d2c741c26267da00a008f379 by Victor Stinner in branch '3.6':
bpo-31009: Move fd_count() to test.support (#7308)
https://github.com/python/cpython/commit/99ba73d1188f4d52d2c741c26267da00a008f379
History
Date User Action Args
2022-04-11 14:58:49adminsetgithub: 75192
2018-06-01 11:47:31vstinnersetmessages: + msg318401
2018-06-01 10:29:11vstinnersetpull_requests: + pull_request6940
2017-07-26 00:10:33vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg299173

stage: resolved
2017-07-25 15:45:57vstinnersetmessages: + msg299115
2017-07-25 12:10:33vstinnersetpull_requests: + pull_request2913
2017-07-25 11:42:38vstinnersetmessages: + msg299050
2017-07-24 09:17:14vstinnersetmessages: + msg298941
title: test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x -> test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x
2017-07-24 09:15:12vstinnercreate