Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_multiprocessing_forkserver leaks references on Python 3 #74958

Closed
vstinner opened this issue Jun 26, 2017 · 14 comments
Closed

test_multiprocessing_forkserver leaks references on Python 3 #74958

vstinner opened this issue Jun 26, 2017 · 14 comments
Labels
3.7 (EOL) end of life performance Performance or resource usage tests Tests in the Lib/test dir

Comments

@vstinner
Copy link
Member

BPO 30775
Nosy @pitrou, @vstinner, @applio
PRs
  • bpo-30775: Fix refleaks in test_multiprocessing #2467
  • [3.6] bpo-30775: Fix refleaks in test_multiprocessing (GH-2467) #2468
  • [3.5] bpo-30775: Fix refleaks in test_multiprocessing (GH-2467) #2469
  • bpo-30775: Clear potential ref cycle between Process and Process target #2470
  • [3.6] bpo-30775: Clear potential ref cycle between Process and Process target (GH-2470) #2471
  • [3.5] bpo-30775: Clear potential ref cycle between Process and Process target (GH-2470) #2472
  • [2.7] bpo-30775: Clear potential ref cycle between Process and Process target (GH-2470) #2473
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-06-28.09:51:49.847>
created_at = <Date 2017-06-26.20:16:57.190>
labels = ['3.7', 'tests', 'performance']
title = 'test_multiprocessing_forkserver leaks references on Python 3'
updated_at = <Date 2017-09-15.16:04:53.679>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2017-09-15.16:04:53.679>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-28.09:51:49.847>
closer = 'pitrou'
components = ['Tests']
creation = <Date 2017-06-26.20:16:57.190>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 30775
keywords = []
message_count = 14.0
messages = ['296942', '296943', '296944', '296945', '296946', '297164', '297166', '297168', '297169', '297170', '297171', '297172', '297174', '302261']
nosy_count = 3.0
nosy_names = ['pitrou', 'vstinner', 'davin']
pr_nums = ['2467', '2468', '2469', '2470', '2471', '2472', '2473']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue30775'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']
    @vstinner
    Copy link
    Member Author 

    $ ./python -m test --match=test.test_multiprocessing_forkserver.WithProcessesTestBarrier.test_barrier --match=test.test_multiprocessing_forkserver.WithProcessesTestFinalize.test_finalize -R 3:3 test_multiprocessing_forkserver

    test_multiprocessing_forkserver leaked [222, 222, 225] memory blocks, sum=669
    test_multiprocessing_forkserver failed

    Total duration: 5 sec
    Tests result: FAILURE

    @vstinner vstinner added tests Tests in the Lib/test dir performance Performance or resource usage 3.7 (EOL) end of life labels Jun 26, 2017
    @vstinner vstinner changed the title [3.6] test_multiprocessing_forkserver leaks references on Python 3.6 test_multiprocessing_forkserver leaks references on Python 3 Jun 26, 2017
    @vstinner
    Copy link
    Member Author

    test_multiprocessing_spawn doesn't leak:

    haypo@selma$ ./python -m test --match=test.test_multiprocessing_spawn.WithProcessesTestBarrier.test_barrier --match=test.test_multiprocessing_spawn.WithProcessesTestFinalize.test_finalize -R 3:3 test_multiprocessing_spawn
    Run tests sequentially
    0:00:00 load avg: 0.11 [1/1] test_multiprocessing_spawn
    beginning 6 repetitions
    123456
    ......
    1 test OK.

    Total duration: 10 sec
    Tests result: SUCCESS

    @vstinner
    Copy link
    Member Author

    The first message was on Python 3.6. On master, regrtest sees even more kinds of leaks: references, memory blocks and file descriptors.

    haypo@selma$ ./python -m test --match=test.test_multiprocessing_forkserver.WithProcessesTestBarrier.test_barrier --match=test.test_multiprocessing_forkserver.WithProcessesTestFinalize.test_finalize -R 3:3 test_multiprocessing_forkserver
    Run tests sequentially
    0:00:00 load avg: 0.25 [1/1] test_multiprocessing_forkserver
    beginning 6 repetitions
    123456
    ......
    test_multiprocessing_forkserver leaked [492, 492, 492] references, sum=1476
    test_multiprocessing_forkserver leaked [221, 222, 225] memory blocks, sum=668
    test_multiprocessing_forkserver leaked [4, 4, 4] file descriptors, sum=12
    test_multiprocessing_forkserver failed

    1 test failed:
    test_multiprocessing_forkserver

    Total duration: 5 sec
    Tests result: FAILURE

    @pitrou
    Copy link
    Member

    pitrou commented Jun 26, 2017

    Is it test_finalize or test_barrier?

    @vstinner
    Copy link
    Member Author

    Is it test_finalize or test_barrier?

    Try yourself: it only leaks when you run both :-)

    @vstinner
    Copy link
    Member Author

    New changeset a79f8fa by Victor Stinner (Antoine Pitrou) in branch 'master':
    bpo-30775: Fix refleaks in test_multiprocessing (bpo-2467)
    a79f8fa

    @vstinner
    Copy link
    Member Author

    Hey, thank you Antoine for fixing the last known reference leak!

    @pitrou
    Copy link
    Member

    pitrou commented Jun 28, 2017

    New changeset e022aad by Antoine Pitrou in branch '3.6':
    [3.6] bpo-30775: Fix refleaks in test_multiprocessing (GH-2467) (bpo-2468)
    e022aad

    @vstinner
    Copy link
    Member Author

    New changeset f15bf1f by Victor Stinner (Antoine Pitrou) in branch '3.5':
    [3.5] bpo-30775: Fix refleaks in test_multiprocessing (GH-2467) (bpo-2469)
    f15bf1f

    @pitrou pitrou closed this as completed Jun 28, 2017
    @vstinner
    Copy link
    Member Author

    New changeset 79d37ae by Victor Stinner (Antoine Pitrou) in branch 'master':
    Clear potential ref cycle between Process and Process target (bpo-2470)
    79d37ae

    @pitrou
    Copy link
    Member

    pitrou commented Jun 28, 2017

    New changeset 38d6a40 by Antoine Pitrou in branch '3.6':
    [3.6] Clear potential ref cycle between Process and Process target (GH-2470) (bpo-2471)
    38d6a40

    @pitrou
    Copy link
    Member

    pitrou commented Jun 28, 2017

    New changeset a9705b7 by Antoine Pitrou in branch '3.5':
    [3.5] Clear potential ref cycle between Process and Process target (GH-2470) (bpo-2472)
    a9705b7

    @vstinner
    Copy link
    Member Author

    New changeset 12536bd by Victor Stinner (Antoine Pitrou) in branch '2.7':
    [2.7] Clear potential ref cycle between Process and Process target (GH-2470) (bpo-2473)
    12536bd

    @vstinner
    Copy link
    Member Author

    The Python 2.7 change caused a regression in Sphinx: bpo-31016.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @hugovk hugovk mentioned this issue May 10, 2023
    Closed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life performance Performance or resource usage tests Tests in the Lib/test dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @pitrou