Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x #75192

Closed
vstinner opened this issue Jul 24, 2017 · 6 comments
Closed

test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x #75192

vstinner opened this issue Jul 24, 2017 · 6 comments
Labels
3.7 (EOL) end of life tests Tests in the Lib/test dir

Comments

@vstinner
Copy link
Member

BPO 31009
Nosy @pitrou, @vstinner, @applio
PRs
  • bpo-31009: Fix support.fd_count() on Windows #2862
  • [3.6] bpo-31009: Move fd_count() to test.support #7308
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-07-26.00:10:33.879>
created_at = <Date 2017-07-24.09:15:12.356>
labels = ['3.7', 'tests']
title = 'test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x'
updated_at = <Date 2018-06-01.11:47:31.649>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2018-06-01.11:47:31.649>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-07-26.00:10:33.879>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-07-24.09:15:12.356>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31009
keywords = ['buildbot']
message_count = 6.0
messages = ['298940', '298941', '299050', '299115', '299173', '318401']
nosy_count = 3.0
nosy_names = ['pitrou', 'vstinner', 'davin']
pr_nums = ['2862', '7308']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue31009'
versions = ['Python 3.7']
    @vstinner
    Copy link
    Member Author

    Same question as usual: why regrtest doesn't display the "running: test_multiprocessing_spawn (319 sec)" update every 30 seconds? Is regrtest main process also stuck?

    http://buildbot.python.org/all/builders/AMD64%20Windows10%203.x/builds/973/steps/test/logs/stdio

    ...
    Run tests in parallel using 2 child processes
    ...
    0:47:28 [403/406] test_buffer passed (119 sec) -- running: test_multiprocessing_spawn (319 sec)
    0:47:30 [404/406] test_turtle passed -- running: test_multiprocessing_spawn (321 sec)
    0:47:32 [405/406] test_dynamic passed -- running: test_multiprocessing_spawn (323 sec)

    command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-x64', '-j2', '--timeout', '900'], attempting to kill
    program finished with exit code 1
    elapsedTime=4059.302000

    @vstinner vstinner added 3.7 (EOL) end of life tests Tests in the Lib/test dir labels Jul 24, 2017
    @vstinner
    Copy link
    Member Author

    Same bug on AMD64 Windows8 3.x buildbot:

    http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/943/steps/test/logs/stdio

    Run tests in parallel using 2 child processes
    ...
    1:03:06 [402/406] test_getargs2 passed -- running: test_multiprocessing_spawn (1203 sec)
    1:03:09 [403/406] test_threadedtempfile passed -- running: test_multiprocessing_spawn (1205 sec)
    1:03:11 [404/406] test_slice passed -- running: test_multiprocessing_spawn (1208 sec)
    1:03:19 [405/406] test_tracemalloc passed -- running: test_multiprocessing_spawn (1215 sec)

    command timed out: 1200 seconds without output running ['Tools\\buildbot\\test.bat', '-x64', '-j2', '--timeout', '900'], attempting to kill
    program finished with exit code 1
    elapsedTime=5005.881000

    @vstinner vstinner changed the title test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x test_multiprocessing_spawn hangs randomly on AMD64 Windows10 3.x and AMD64 Windows8 3.x Jul 24, 2017
    @vstinner
    Copy link
    Member Author

    test_multiprocessing_spawn hangs on running test_child_fd_inflation(). Child processes call test.support.fd_count(). But in a debug build, if the fd is invalid, os.dup(fd) kills immediately the process without letting Python to handle the error. It seems like os.dup(fd) raises a OSError(EBADF), as expected, in release mode.

    The test is new and currently only in master, see bpo-26732. The commit:

    commit 896145d
    Author: Antoine Pitrou <pitrou@free.fr>
    Date: Sat Jul 22 13:22:54 2017 +0200

    bpo-26732: fix too many fds in processes started with the "forkserver" method (bpo-2813)

* bpo-26732: fix too many fds in processes started with the "forkserver" method

A child process would inherit as many fds as the number of still-running children.

* Add blurb and test comment

    @vstinner
    Copy link
    Member Author

    New changeset 302bbbe by Victor Stinner in branch 'master':
    bpo-31009: Fix support.fd_count() on Windows (bpo-2862)
    302bbbe

    @vstinner
    Copy link
    Member Author

    Windows buildbots are green again, I close the issue.

    @vstinner
    Copy link
    Member Author

    vstinner commented Jun 1, 2018

    New changeset 99ba73d by Victor Stinner in branch '3.6':
    bpo-31009: Move fd_count() to test.support (bpo-7308)
    99ba73d

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life tests Tests in the Lib/test dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner