Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_asyncio leaks dangling threads #75433

Closed
vstinner opened this issue Aug 21, 2017 · 7 comments
Closed

test_asyncio leaks dangling threads #75433

vstinner opened this issue Aug 21, 2017 · 7 comments
Labels
3.7 (EOL) end of life performance Performance or resource usage tests Tests in the Lib/test dir topic-asyncio

Comments

@vstinner
Copy link
Member

BPO 31250
Nosy @vstinner, @1st1
PRs
  • bpo-31250, test_asyncio: fix dangling threads #3252
  • bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() #3264
  • [3.6] bpo-31250: test_asyncio: fix dangling threads #3517
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-09-12.21:18:39.212>
created_at = <Date 2017-08-21.16:33:56.001>
labels = ['3.7', 'expert-asyncio', 'tests', 'performance']
title = 'test_asyncio leaks dangling threads'
updated_at = <Date 2017-09-13.09:56:51.402>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2017-09-13.09:56:51.402>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-09-12.21:18:39.212>
closer = 'vstinner'
components = ['Tests', 'asyncio']
creation = <Date 2017-08-21.16:33:56.001>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31250
keywords = ['patch']
message_count = 7.0
messages = ['300637', '301048', '301097', '301099', '301131', '301155', '301995']
nosy_count = 2.0
nosy_names = ['vstinner', 'yselivanov']
pr_nums = ['3252', '3264', '3517']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue31250'
versions = ['Python 3.6', 'Python 3.7']
    @vstinner
    Copy link
    Member Author

    Using PR 3138 of bpo-31234, I noticed that test_asyncio leaks dangling threads.

    Tests which randomly leak threads:
    ---
    test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_no_inet_pton
    test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_service_name
    test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_bluetooth
    test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_datagram_endpoint_no_addrinfo
    test.test_asyncio.test_futures.DuckTests.test_wrap_future
    test.test_asyncio.test_futures.CFutureTests.test_wrap_future
    test.test_asyncio.test_futures.CFutureTests.test_wrap_future_use_global_loop
    test.test_asyncio.test_futures.PyFutureTests.test_wrap_future
    test.test_asyncio.test_futures.PyFutureTests.test_wrap_future_use_global_loop
    test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests.test_sock_connect_resolve_using_socket_params
    test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe
    test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_with_timeout
    test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_cancelled
    test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_factory_exception
    ---

    Write this list into "tests" and run "python3 -m test --matchfile=tests".

    haypo@selma$ ./python -u -m test -v test_asyncio --matchfile=tests
    == CPython 3.7.0a0 (heads/concurrent_futures:f277fa3, Aug 21 2017, 18:11:41) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]
    (...)
    test_create_connection_bluetooth (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... ok
    test_create_connection_no_inet_pton (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
    ok
    test_create_connection_service_name (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
    ok
    test_create_datagram_endpoint_no_addrinfo (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_wrap_future (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_wrap_future_use_global_loop (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_wrap_future (test.test_asyncio.test_futures.DuckTests) ... ok
    test_wrap_future (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_wrap_future_use_global_loop (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_sock_connect_resolve_using_socket_params (test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_run_coroutine_threadsafe (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
    Test coroutine submission from a thread to an event loop. ... ok
    test_run_coroutine_threadsafe_task_cancelled (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
    Test coroutine submission from a tread to an event loop ... ok
    test_run_coroutine_threadsafe_task_factory_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
    Test coroutine submission from a tread to an event loop ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
    ok
    test_run_coroutine_threadsafe_with_timeout (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
    Test coroutine submission from a thread to an event loop ... ok
    (...)

    @vstinner vstinner added 3.7 (EOL) end of life tests Tests in the Lib/test dir topic-asyncio performance Performance or resource usage labels Aug 21, 2017
    @vstinner
    Copy link
    Member Author

    I proposed PR 3252 to only fix unit tests, but not asyncio.

    An alternative is to modify BaseEventLoop.close() to replace executor.shutdown(wait=False) with executor.shutdown(wait=True), but it would change the behaviour. I'm not sure if we want to wait until all threads complete on close()?

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 1, 2017

    New changeset 16432be by Victor Stinner in branch 'master':
    bpo-31250, test_asyncio: fix dangling threads (bpo-3252)
    16432be

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 1, 2017

    I picked the safe option: only modify unit tests.

    @vstinner vstinner closed this as completed Sep 1, 2017
    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 1, 2017

    Hum, EventLoopTestsMixin.tearDown() still has a bug. I reopen the issue.

    @vstinner vstinner reopened this Sep 1, 2017
    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 1, 2017

    New changeset e8a533f by Victor Stinner in branch 'master':
    bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() (bpo-3264)
    e8a533f

    @vstinner
    Copy link
    Member Author

    New changeset 5013a5e by Victor Stinner in branch '3.6':
    [3.6] bpo-31250: test_asyncio: fix dangling threads (bpo-3517)
    5013a5e

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life performance Performance or resource usage tests Tests in the Lib/test dir topic-asyncio
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @vstinner