Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crash with unbounded recursion in except statement #86666

Closed
xxm mannequin opened this issue Nov 29, 2020 · 12 comments
Closed

crash with unbounded recursion in except statement #86666

xxm mannequin opened this issue Nov 29, 2020 · 12 comments
Assignees
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@xxm
Copy link
Mannequin

xxm mannequin commented Nov 29, 2020

BPO 42500
Nosy @gpshead, @ronaldoussoren, @ericvsmith, @ambv, @markshannon, @serhiy-storchaka, @vedgar
PRs
  • bpo-42500: Fix recursion in or after except #23568
  • [3.9] bpo-42500: Fix recursion in or after except (GH-23568) #24501
  • [3.9] bpo-43710: Rollback the 3.9 bpo-42500 fix, it broke the ABI in 3.9.3 #25179
  • Superseder
  • bpo-45806: Cannot Recover From StackOverflow in 3.9 Tests
  • Files
  • error1.py: run this program by using command " python error1.py" under Ubuntu
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/markshannon'
    closed_at = <Date 2020-12-02.13:54:51.328>
    created_at = <Date 2020-11-29.07:39:23.532>
    labels = ['interpreter-core', '3.9', 'type-crash']
    title = 'crash with unbounded recursion in except statement'
    updated_at = <Date 2021-11-19.18:16:43.116>
    user = 'https://bugs.python.org/xxm'

    bugs.python.org fields:

    activity = <Date 2021-11-19.18:16:43.116>
    actor = 'lukasz.langa'
    assignee = 'Mark.Shannon'
    closed = True
    closed_date = <Date 2020-12-02.13:54:51.328>
    closer = 'Mark.Shannon'
    components = ['Interpreter Core']
    creation = <Date 2020-11-29.07:39:23.532>
    creator = 'xxm'
    dependencies = []
    files = ['49635']
    hgrepos = []
    issue_num = 42500
    keywords = ['patch']
    message_count = 12.0
    messages = ['382042', '382055', '382057', '382059', '382069', '382070', '382113', '382115', '382138', '382311', '387905', '390182']
    nosy_count = 8.0
    nosy_names = ['gregory.p.smith', 'ronaldoussoren', 'eric.smith', 'lukasz.langa', 'Mark.Shannon', 'serhiy.storchaka', 'veky', 'xxm']
    pr_nums = ['23568', '24501', '25179']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = '45806'
    type = 'crash'
    url = 'https://bugs.python.org/issue42500'
    versions = ['Python 3.9']

    @xxm
    Copy link
    Mannequin Author

    xxm mannequin commented Nov 29, 2020

    This program can work well on Python 3.5.2 and Python2.7 with the following output.
    "Invalid Entry, try again"

    However it will crash on Python3.9.0rc1, 3.10.0a2 with the following error message:
    "
    Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.
    Python runtime state: initialized

    Current thread 0x00007fc4b679b700 (most recent call first):
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 38 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
    ...
    Aborted (core dumped)
    "

    @xxm xxm mannequin added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump labels Nov 29, 2020
    @ronaldoussoren
    Copy link
    Contributor

    The crash is not nice, but....

    The script contains the following line in the definition of status():

    except status() as e:
    

    That will cause unbounded recursion when os.fstat raises an exception, which likely happens here. You probably want to catch OSError (or os.error) instead.

    This scriptlet is a smaller reproduction of the problem:

    def status():

    try:
    1/0

    except status():
    pass

    status()

    @ronaldoussoren ronaldoussoren changed the title status() crashes on Python3.9 and 3.10 crash with unbounded recursion in except statement Nov 29, 2020
    @ronaldoussoren ronaldoussoren changed the title status() crashes on Python3.9 and 3.10 crash with unbounded recursion in except statement Nov 29, 2020
    @vedgar
    Copy link
    Mannequin

    vedgar mannequin commented Nov 29, 2020

    Recursion limit is probably set too high, but nonetheless, the program is nonsensical, the line 39 especially.

    The reason for difference in behavior is that RecursionError is caught by blank except in the earlier Pythons, while it crashes the stack in later ones. In any case, it's almost certain this isn't what you wanted to write.

    (And yet another reason to remove blank except from Python: it abuse to correct use ratio is probably over two orders of magnitude.)

    @ronaldoussoren
    Copy link
    Contributor

    The value of the recursion limit is not relevant here, I get the same crash when I set the recursion limit to 100 (in my reproducer scriptlet).

    There might be a missing recursion level check in the exception handling machinery though, self-recursion in an except-clause is rather unusual. But that's just a blind guess, I haven't though much about what is supposed to happen here.

    @ericvsmith
    Copy link
    Member

    Here's the smallest reproducer I could come up with. It fails on Windows with 3.9 native(compiled locally) (Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow), works (raises RecursionError) with cygwin 3.8.3.

    import os
    
    def status():
        for fd in range(4):
            try:
                st = os.fstat(fd)
            except status() as e:
                pass
    
    status()

    @ericvsmith
    Copy link
    Member

    Note that changing the os.fstat line to just "raise OSError()" no longer causes the "Fatal Python error", but rather gives the expected recursion exception.

    Here's a shorter version that causes the fatal error in Windows native 3.9, cygwin 3.8.3, and Fedora Linux 3.7.7. So this isn't new with 3.9.

    import os
    
    def status():
        try:
            st = os.fstat(4)
        except status() as e:
            pass
    
    status()

    @ronaldoussoren
    Copy link
    Contributor

    As mentioned earlier I can reproduce this without calling os.fstat (python 3.9.1rc, macOS 10.15):

    # ---

    def status():
       try:
          1/0

    except status():
    pass

    status()
    # 

    The crash happens both with the default recursion limit as well with a recursion limit of 10. Setting the recursion limit doesn't seem to have an effect on what's happening (in both cases I get a long traceback).

    @ronaldoussoren
    Copy link
    Contributor

    See also bpo-42509

    @markshannon
    Copy link
    Member

    Ronald's test case also causes a Fatal Error on master.

    2.7.18 produces the expected "RuntimeError: maximum recursion depth exceeded"

    @markshannon
    Copy link
    Member

    New changeset 4e7a69b by Mark Shannon in branch 'master':
    bpo-42500: Fix recursion in or after except (GH-23568)
    4e7a69b

    @ambv
    Copy link
    Contributor

    ambv commented Mar 2, 2021

    New changeset 8b795ab by Mark Shannon in branch '3.9':
    bpo-42500: Fix recursion in or after except (GH-23568) (bpo-24501)
    8b795ab

    @ambv
    Copy link
    Contributor

    ambv commented Apr 4, 2021

    New changeset c7b0fec by Gregory P. Smith in branch '3.9':
    [3.9] bpo-43710: Rollback the 3.9 bpo-42500 fix, it broke the ABI in 3.9.3 (bpo-25179)
    c7b0fec

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants