classification
Title: crash with unbounded recursion in except statement
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Mark.Shannon, eric.smith, gregory.p.smith, lukasz.langa, ronaldoussoren, serhiy.storchaka, veky, xxm
Priority: normal Keywords: patch

Created on 2020-11-29 07:39 by xxm, last changed 2021-04-04 11:02 by lukasz.langa. This issue is now closed.

Files
File name Uploaded Description Edit
error1.py xxm, 2020-11-29 07:39 run this program by using command " python error1.py" under Ubuntu
Pull Requests
URL Status Linked Edit
PR 23568 merged Mark.Shannon, 2020-11-30 14:41
PR 24501 merged Mark.Shannon, 2021-02-10 13:49
PR 25179 merged gregory.p.smith, 2021-04-04 04:17
Messages (12)
msg382042 - (view) Author: Xinmeng Xia (xxm) Date: 2020-11-29 07:39
This program can work well on Python 3.5.2 and Python2.7 with the following output.
"Invalid Entry, try again"


However it will crash on Python3.9.0rc1, 3.10.0a2 with the following error message:
"
Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.
Python runtime state: initialized

Current thread 0x00007fc4b679b700 (most recent call first):
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 38 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  File "/home/xxm/Desktop/methodfuzzer/error/error1.py", line 39 in status
  ...
Aborted (core dumped)
"
msg382055 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-29 12:01
The crash is not nice, but....

The script contains the following line in the definition of status():

    except status() as e:

That will cause unbounded recursion when os.fstat raises an exception, which likely happens here. You probably want to catch OSError (or os.error) instead.


This scriptlet is a smaller reproduction of the problem:

def status():

   try:
      1/0

   except status():
      pass

status()
msg382057 - (view) Author: Vedran Čačić (veky) * Date: 2020-11-29 12:08
Recursion limit is probably set too high, but nonetheless, the program is nonsensical, the line 39 especially.

The reason for difference in behavior is that RecursionError is caught by blank except in the earlier Pythons, while it crashes the stack in later ones. In any case, it's almost certain this isn't what you wanted to write.

(And yet another reason to remove blank except from Python: it abuse to correct use ratio is probably over two orders of magnitude.)
msg382059 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-29 12:16
The value of the recursion limit is not relevant here, I get the same crash when I set the recursion limit to 100 (in my reproducer scriptlet).

There might be a missing recursion level check in the exception handling machinery though, self-recursion in an except-clause is rather unusual. But that's just a blind guess, I haven't though much about what is supposed to happen here.
msg382069 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-11-29 15:05
Here's the smallest reproducer I could come up with. It fails on Windows with 3.9 native(compiled locally) (Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow), works (raises RecursionError) with cygwin 3.8.3.

import os

def status():
    for fd in range(4):
        try:
            st = os.fstat(fd)
        except status() as e:
            pass

status()
msg382070 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-11-29 15:12
Note that changing the os.fstat line to just "raise OSError()" no longer causes the "Fatal Python error", but rather gives the expected recursion exception.

Here's a shorter version that causes the fatal error in Windows native 3.9, cygwin 3.8.3, and Fedora Linux 3.7.7. So this isn't new with 3.9.

import os

def status():
    try:
        st = os.fstat(4)
    except status() as e:
        pass

status()
msg382113 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-30 08:00
As mentioned earlier I can reproduce this without calling os.fstat (python 3.9.1rc, macOS 10.15):

# ---
def status():
   try:
      1/0

   except status():
      pass

status()
# ----

The crash happens both with the default recursion limit as well with a recursion limit of 10. Setting the recursion limit doesn't seem to have an effect on what's happening (in both cases I get a long traceback).
msg382115 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-30 08:11
See also #42509
msg382138 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-11-30 10:18
Ronald's test case also causes a Fatal Error on master.

2.7.18 produces the expected "RuntimeError: maximum recursion depth exceeded"
msg382311 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-12-02 13:31
New changeset 4e7a69bdb63a104587759d7784124492dcdd496e by Mark Shannon in branch 'master':
bpo-42500: Fix recursion in or after except (GH-23568)
https://github.com/python/cpython/commit/4e7a69bdb63a104587759d7784124492dcdd496e
msg387905 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-03-02 10:37
New changeset 8b795ab5541d8a4e69be4137dfdc207714270b77 by Mark Shannon in branch '3.9':
bpo-42500: Fix recursion in or after except (GH-23568) (#24501)
https://github.com/python/cpython/commit/8b795ab5541d8a4e69be4137dfdc207714270b77
msg390182 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-04-04 11:02
New changeset c7b0feca25fc68ec3e0884b82e5f45a4da011e8e by Gregory P. Smith in branch '3.9':
[3.9] bpo-43710: Rollback the 3.9 bpo-42500 fix, it broke the ABI in 3.9.3 (#25179)
https://github.com/python/cpython/commit/c7b0feca25fc68ec3e0884b82e5f45a4da011e8e
History
Date User Action Args
2021-04-04 11:02:51lukasz.langasetmessages: + msg390182
2021-04-04 04:17:53gregory.p.smithsetnosy: + gregory.p.smith

pull_requests: + pull_request23921
2021-03-02 10:37:11lukasz.langasetnosy: + lukasz.langa
messages: + msg387905
2021-02-17 12:49:33iritkatriellinkissue28179 superseder
2021-02-10 13:49:19Mark.Shannonsetpull_requests: + pull_request23290
2020-12-22 08:32:50ronaldoussorenlinkissue42652 superseder
2020-12-22 08:32:24ronaldoussorenlinkissue42651 superseder
2020-12-02 13:54:51Mark.Shannonsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-12-02 13:31:09Mark.Shannonsetmessages: + msg382311
2020-11-30 14:41:19Mark.Shannonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request22449
2020-11-30 10:18:26Mark.Shannonsetassignee: Mark.Shannon
messages: + msg382138
2020-11-30 08:11:12ronaldoussorensetmessages: + msg382115
2020-11-30 08:00:26ronaldoussorensetmessages: + msg382113
2020-11-29 15:12:52eric.smithsetmessages: + msg382070
2020-11-29 15:05:00eric.smithsetnosy: + eric.smith
messages: + msg382069
2020-11-29 14:36:57serhiy.storchakasetnosy: + Mark.Shannon, serhiy.storchaka
2020-11-29 12:16:49ronaldoussorensetmessages: + msg382059
2020-11-29 12:08:22vekysetnosy: + veky
messages: + msg382057
2020-11-29 12:02:49ronaldoussorensettitle: status() crashes on Python3.9 and 3.10 -> crash with unbounded recursion in except statement
2020-11-29 12:01:48ronaldoussorensetnosy: + ronaldoussoren
messages: + msg382055
2020-11-29 07:39:23xxmcreate