Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Queue with maxsize can lead to deadlocks #77259

Closed
tomMoral mannequin opened this issue Mar 15, 2018 · 7 comments
Closed

Queue with maxsize can lead to deadlocks #77259

tomMoral mannequin opened this issue Mar 15, 2018 · 7 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tomMoral
Copy link
Mannequin

tomMoral mannequin commented Mar 15, 2018

BPO 33078
Nosy @pitrou, @ned-deily, @applio, @tomMoral
PRs
  • bpo-33078 - FIX queue size on pickling error #6119
  • [3.7] bpo-33078 - Fix queue size on pickling error (GH-6119) #6178
  • bpo-33078 FIX failure on OSX sem_getvalue #6180
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-03-21.19:01:57.839>
created_at = <Date 2018-03-15.06:36:54.012>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'Queue with maxsize can lead to deadlocks'
updated_at = <Date 2018-03-21.21:27:09.127>
user = 'https://github.com/tomMoral'

    bugs.python.org fields:

    activity = <Date 2018-03-21.21:27:09.127>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2018-03-21.19:01:57.839>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2018-03-15.06:36:54.012>
creator = 'tomMoral'
dependencies = []
files = []
hgrepos = []
issue_num = 33078
keywords = ['patch']
message_count = 7.0
messages = ['313855', '314208', '314210', '314211', '314219', '314221', '314224']
nosy_count = 4.0
nosy_names = ['pitrou', 'ned.deily', 'davin', 'tomMoral']
pr_nums = ['6119', '6178', '6180']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33078'
versions = ['Python 3.7', 'Python 3.8']
    @tomMoral
    Copy link
    Mannequin Author

    tomMoral mannequin commented Mar 15, 2018

    The fix for the Queue._feeder does not properly handle the size of the Queue. This can lead to a situation where the Queue is considered as Full when it is empty. Here is a reproducing script:

    import multiprocessing as mp

q = mp.Queue(1)


class FailPickle():
    def __reduce__(self):
        raise ValueError()

q.put(FailPickle())
print("Queue is full:", q.full())
q.put(0)
print(f"Got result: {q.get()}")

    @tomMoral tomMoral mannequin added 3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 15, 2018
    @pitrou
    Copy link
    Member

    pitrou commented Mar 21, 2018

    New changeset e2f33ad by Antoine Pitrou (Thomas Moreau) in branch 'master':
    bpo-33078 - Fix queue size on pickling error (GH-6119)
    e2f33ad

    @pitrou
    Copy link
    Member

    pitrou commented Mar 21, 2018

    New changeset bb5b529 by Antoine Pitrou (Miss Islington (bot)) in branch '3.7':
    bpo-33078 - Fix queue size on pickling error (GH-6119) (GH-6178)
    bb5b529

    @pitrou pitrou closed this as completed Mar 21, 2018
    @ned-deily
    Copy link
    Member

    Buildbots are failing on macOS:

    http://buildbot.python.org/all/#/builders/14/builds/804/steps/4/logs/stdio
    http://buildbot.python.org/all/#/builders/93/builds/459/steps/4/logs/stdio

    @ned-deily ned-deily reopened this Mar 21, 2018
    @ned-deily
    Copy link
    Member

    PR 6180 appears to have fixed the 3.x failures, thanks!

    @pitrou
    Copy link
    Member

    pitrou commented Mar 21, 2018

    Good, I think we can close again now!

    @pitrou pitrou closed this as completed Mar 21, 2018
    @ned-deily
    Copy link
    Member

    (Fot the record, PR 6181 was the 3.7 backport of PR 6180.)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @pitrou @ned-deily