Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ProcessPoolExecutor hangs on shutdown nowait with pickling failure #83285

Closed
tomMoral mannequin opened this issue Dec 20, 2019 · 5 comments
Closed

ProcessPoolExecutor hangs on shutdown nowait with pickling failure #83285

tomMoral mannequin opened this issue Dec 20, 2019 · 5 comments
Labels
3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tomMoral
Copy link
Mannequin

tomMoral mannequin commented Dec 20, 2019

BPO 39104
Nosy @brianquinlan, @pitrou, @vstinner, @tomMoral, @miss-islington, @rad-pat
PRs
  • bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure #17670
    		•
    Files
  • main.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-02-16.18:10:37.153>
created_at = <Date 2019-12-20.06:52:21.989>
labels = ['type-bug', 'library', '3.9']
title = 'ProcessPoolExecutor hangs on shutdown nowait with pickling failure'
updated_at = <Date 2020-04-27.22:08:26.824>
user = 'https://github.com/tomMoral'

    bugs.python.org fields:

    activity = <Date 2020-04-27.22:08:26.824>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-02-16.18:10:37.153>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2019-12-20.06:52:21.989>
creator = 'tomMoral'
dependencies = []
files = ['48794']
hgrepos = []
issue_num = 39104
keywords = ['patch']
message_count = 5.0
messages = ['358697', '360051', '362088', '362089', '367466']
nosy_count = 6.0
nosy_names = ['bquinlan', 'pitrou', 'vstinner', 'tomMoral', 'miss-islington', 'patbuxton']
pr_nums = ['17670']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39104'
versions = ['Python 3.9']
    @tomMoral
    Copy link
    Mannequin Author

    tomMoral mannequin commented Dec 20, 2019

    The attached scripts hangs on python3.7+.
    This is due to the fact that the main process closes the communication channels directly while the queue_management_thread might still use them.

    To prevent that, all the closing should be handled by the queue_management_thread.

    @tomMoral tomMoral mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir labels Dec 20, 2019
    @rad-pat
    Copy link
    Mannequin

    rad-pat mannequin commented Jan 15, 2020

    Added core developer to get some feedback

    @miss-islington
    Copy link
    Contributor

    New changeset a5cbab5 by Thomas Moreau in branch 'master':
    bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)
    a5cbab5

    @pitrou
    Copy link
    Member

    pitrou commented Feb 16, 2020

    Thank you Thomas. Ii'd rather not backport delicate changes, so closing now.

    @pitrou pitrou removed 3.7 (EOL) end of life 3.8 only security fixes labels Feb 16, 2020
    @pitrou pitrou closed this as completed Feb 16, 2020
    @pitrou pitrou added type-bug An unexpected behavior, bug, or error and removed 3.7 (EOL) end of life 3.8 only security fixes labels Feb 16, 2020
    @pitrou pitrou closed this as completed Feb 16, 2020
    @pitrou pitrou added the type-bug An unexpected behavior, bug, or error label Feb 16, 2020
    @vstinner
    Copy link
    Member

    bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling failure (GH-17670)
    a5cbab5

    ProcessPoolSpawnProcessPoolExecutorTest.test_killed_child() of test_concurrent_futures started to fail randomly since this change: see bpo-39995.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @nkoep nkoep mentioned this issue Apr 12, 2022
    Open
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @pitrou @miss-islington