Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Occasional RuntimeError from Condition.notify #66381

Closed
dougz mannequin opened this issue Aug 11, 2014 · 5 comments
Closed

Occasional RuntimeError from Condition.notify #66381

dougz mannequin opened this issue Aug 11, 2014 · 5 comments
Labels
stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@dougz
Copy link
Mannequin

dougz mannequin commented Aug 11, 2014

BPO 22185
Nosy @tim-one, @rhettinger, @pitrou, @vstinner
Files
  • fix.diff: move self._waiters modification inside lock
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2014-08-29.21:28:26.728>
    created_at = <Date 2014-08-11.23:47:11.131>
    labels = ['library', 'type-crash']
    title = 'Occasional RuntimeError from Condition.notify'
    updated_at = <Date 2014-08-29.21:28:26.726>
    user = 'https://bugs.python.org/dougz'

    bugs.python.org fields:

    activity = <Date 2014-08-29.21:28:26.726>
    actor = 'pitrou'
    assignee = 'none'
    closed = True
    closed_date = <Date 2014-08-29.21:28:26.728>
    closer = 'pitrou'
    components = ['Library (Lib)']
    creation = <Date 2014-08-11.23:47:11.131>
    creator = 'dougz'
    dependencies = []
    files = ['36351']
    hgrepos = []
    issue_num = 22185
    keywords = ['patch']
    message_count = 5.0
    messages = ['225208', '225213', '226068', '226088', '226089']
    nosy_count = 6.0
    nosy_names = ['tim.peters', 'rhettinger', 'pitrou', 'vstinner', 'python-dev', 'dougz']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue22185'
    versions = ['Python 3.4', 'Python 3.5']

    @dougz
    Copy link
    Mannequin Author

    dougz mannequin commented Aug 11, 2014

    Condition.wait() modifies self._waiters without holding the lock (when a wait with timeout times out without the condition being notified).

    If this happens to occur in between construction of the _islice and _deque objects in Condition.notify():

        def notify(self, n=1):
            [...]
            all_waiters = self._waiters
            waiters_to_notify = _deque(_islice(all_waiters, n))

    then the result is a RuntimeError exception:

    File "/usr/lib/python3.4/threading.py", line 358, in notify_all
      self.notify(len(self.\_waiters))
    File "/usr/lib/python3.4/threading.py", line 341, in notify
      waiters_to_notify = \_deque(_islice(all_waiters, n))
    

    RuntimeError: deque mutated during iteration

    (I have a server which makes extensive use of conditions on which this happens about once a day.)

    This patch fixes this bug by moving wait()'s modification of self._waiters to be inside the lock, as suggested by Antoine Pitrou here: http://bugs.python.org/issue17385#msg183875

    @dougz dougz mannequin added stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump labels Aug 11, 2014
    @tim-one
    Copy link
    Member

    tim-one commented Aug 12, 2014

    +1. I agree it's a bug, that the diagnosis is correct, and that the patch will fix it :-)

    @dougz
    Copy link
    Mannequin Author

    dougz mannequin commented Aug 29, 2014

    So, what happens now? What do I need to do to make progress on this?

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 29, 2014

    New changeset 4cce39cfe46c by Antoine Pitrou in branch '3.4':
    Issue bpo-22185: Fix an occasional RuntimeError in threading.Condition.wait() caused by mutation of the waiters queue without holding the lock.
    http://hg.python.org/cpython/rev/4cce39cfe46c

    New changeset 78a38f8bd5d9 by Antoine Pitrou in branch 'default':
    Issue bpo-22185: Fix an occasional RuntimeError in threading.Condition.wait() caused by mutation of the waiters queue without holding the lock.
    http://hg.python.org/cpython/rev/78a38f8bd5d9

    @pitrou
    Copy link
    Member

    pitrou commented Aug 29, 2014

    It only needed someone to push your patch, which I just did.
    Thank you very much Doug, your contribution is appreciated!

    @pitrou pitrou closed this as completed Aug 29, 2014
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants