classification
Title: deadlock with asyncio+contextmanager+ExitStack
Type: behavior Stage:
Components: asyncio Versions: Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder: CPython hangs on error __context__ set to the error itself
View: 25782
Assigned To: yselivanov Nosy List: gvanrossum, ncoghlan, oconnor663, vstinner, yselivanov
Priority: normal Keywords: 3.5regression

Created on 2015-12-02 15:37 by oconnor663, last changed 2015-12-02 20:42 by yselivanov. This issue is now closed.

Messages (13)
msg255719 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-12-02 15:37
The following hangs at 100% CPU on Python 3.5, though not on Python 3.4:

1) Start an asyncio coroutine with run_until_complete().
2) Inside the coroutine, enter an ExitStack using a with-statement.
3) Inside the with-statement, call ExitStack.enter_context() with a generator context manager. It doesn't matter what the generator yields.
4) After the enter_context() call, raise an exception.

Here's an example script that does all of this and repros the hang: https://gist.github.com/oconnor663/483db2820bb5f877c9ed
msg255721 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-12-02 16:15
Interestingly, it doesn't hang when you raise a different error. There's some new code dealing with the RuntimeError coming out of a generator if it raises StopIteration (instead of returning) introduced by issue #22906. Yury, it looks like you introduced that? (The diff is this:

changeset:   95932:36a8d935c322
user:        Yury Selivanov <yselivanov@sprymix.com>
date:        Sat May 09 11:44:30 2015 -0400
summary:     PEP 479: Change StopIteration handling inside generators.
msg255722 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 16:30
Trying to reproduce without contextstack.
msg255726 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 16:59
Here's a minimal test to reproduce:

    import reprlib


    def main():
        if 0:
            yield
        raise RuntimeError


    m = main()
    try:
        m.send(None)
    except RuntimeError as ex:
        ex.__context__ = ex
        reprlib.repr(ex)
        

Looks like it's a bug in reprlib.  It's not related to PEP 492/479.  

It's also reproducible in Python 3.4 and 3.3.

Nick, ExitStack does this (indirectly) 'ex.__context__ = ex' thing -- I think that's a bug of contextlib.
msg255728 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 17:03
Created another issue for the reprlib bug: issue 25781.  It appears we don't even need a generator:

    import reprlib

    try:
        raise RuntimeError
    except RuntimeError as ex:
        ex.__context__ = ex
        reprlib.repr(ex)

Closing this one with "not a bug".
msg255730 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 17:12
It's not even a reprlib bug:

try:
    raise Exception
except Exception as ex:
    ex.__context__ = ex
    hasattr(1, 'aa')
msg255732 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-12-02 17:14
Thanks for chasing this down. Yury, can you suggest a workaround?
msg255733 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 17:18
> Thanks for chasing this down. Yury, can you suggest a workaround?

I'm not sure how to workaround this :(  Hopefully we can fix this in 3.5.1.
msg255748 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 18:33
FWIW the bug was identified in issue 25782.  I've drafted a patch to fix it, please review.
msg255749 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 18:35
The question is whether we should raise an exception or not:

   ex.__context__ = ex
msg255765 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 20:00
Another issue for contextlib: http://bugs.python.org/issue25786
msg255777 - (view) Author: Jack O'Connor (oconnor663) * Date: 2015-12-02 20:40
Yury, can you help me understand why `hasattr("foo", "bar")` triggers the infinite loop there, but not `print("foo")`?
msg255778 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-12-02 20:42
> Yury, can you help me understand why `hasattr("foo", "bar")` triggers the infinite loop there, but not `print("foo")`?


hasattr uses getattr under the hood. getattr raises an AttributeError, and that triggers PyErr_SetError, which has an infinite "while" loop.  Instead of "hasattr" you can use anything that raises an error.
History
Date User Action Args
2015-12-02 20:42:08yselivanovsetmessages: + msg255778
2015-12-02 20:40:27oconnor663setmessages: + msg255777
2015-12-02 20:00:52yselivanovsetmessages: + msg255765
2015-12-02 18:35:31yselivanovsetmessages: + msg255749
2015-12-02 18:33:28yselivanovsetmessages: + msg255748
2015-12-02 17:18:10yselivanovsetmessages: + msg255733
2015-12-02 17:14:34oconnor663setmessages: + msg255732
2015-12-02 17:14:19yselivanovsetsuperseder: infinite loop in reprlib -> CPython hangs on error __context__ set to the error itself
2015-12-02 17:12:49yselivanovsetmessages: + msg255730
2015-12-02 17:03:22yselivanovsetstatus: open -> closed
superseder: infinite loop in reprlib
resolution: not a bug
messages: + msg255728
2015-12-02 16:59:02yselivanovsetmessages: + msg255726
2015-12-02 16:30:26yselivanovsetnosy: + ncoghlan
messages: + msg255722
2015-12-02 16:15:54gvanrossumsetkeywords: + 3.5regression
assignee: yselivanov
messages: + msg255721
2015-12-02 15:37:25oconnor663create