Title: RecursionError not caught explicitly may crash with "Aborted (core dumped)"
Type: crash Stage: resolved
Components: Versions:
Status: closed Resolution: duplicate
Dependencies: Superseder: Infinite recursion tests triggering a segfault
View: 18075
Assigned To: Nosy List: Delgan, ned.deily
Priority: normal Keywords:

Created on 2019-06-21 16:08 by Delgan, last changed 2019-06-22 08:12 by Delgan. This issue is now closed.

Messages (3)
msg346232 - (view) Author: Delgan (Delgan) * Date: 2019-06-21 16:08
This is an issue that I first saw repoted by @remi.lapeyre on issue36272:
It also be reported some years ago:
I searched but I did not find any existing issue, so I open a new one, I hope this is not a duplicate.

It's easily reproducible:

def rec():
    except Exception:

It will kill the CPython interpreter (@remi.lapeyre provided explanation in the issue I linked).

Of course, this is a contrived example. Still, I guess CPython should avoid to crash in any situation. This is a tricky edge case: by doing `except Exception:` one could expect to correctly handle all kind of errors without being aware that in some situation the "RecursionError" need to be managed too.

Particularly, the fix in issue36272 solely patches "StreamHandler":
There is many other handlers that are still affected, every time you see "except Exception" in "emit()":

Finally, it should also be noted that the "except RecursionError" workaround implies that loggers are not longer able to catch recursive erros in "write()" or "format()" without re-raising the RecursionError and (probably) halting the program.
It may happen for example that a faulty implementation of a custom Handler or LogRecord attribute cause a RecursionError. Thanks to the "raiseExceptions = false" logging option, this would usually be caught and a message printed on stderr without killing the app.
msg346242 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-21 18:35
Thank you for the report.  The issue of the interpreter not being able to gracefully handle a stack overflow error is an old one: see, for example Issue18075 or Issue35542.  So far, no one has proposed a solution and, as can be seen in Issue18075, workarounds someimes produce unexpected side-effects that can be more problematic than the stack overflow segfault.  I'm going to close this as a duplicate of Issue18075.  If anyone has further suggestions for solutions, please continue the discussion there.
msg346271 - (view) Author: Delgan (Delgan) * Date: 2019-06-22 08:12
Thanks to your answer, @ned.deily.

Sorry to insist, but are you sure this is a duplicate of issue18075 / issue35542?

According to the discussion in issue36272, it seems this is not related to stack exhaustion nor to issue18075.

Also, @remi.lapeyre suggested that a fix could be possible by modifying ceval.c. (re-raise RecursionError if it has been cleared).

issue18075 and issue35542 are both of status "closed", I would expect this issue to be duplicate to an "open" issue because there is a clear reproducible "bug" which crashes the interpreter.

There is also the problems still remaining in logging Handlers that I linked.

I think this is an issue with how recursion error are handled in ceval.c, this is related to "tstate->overflowed" and "tstate->recursion_depth > recursion_limit + 50".
If I understand correctly, this is implemented like this to allow gracefully catch and handle RecursionError from the calling function.
For example, why not re-raise the exception if the first one could not be handled correctly?
Date User Action Args
2019-06-22 08:12:39Delgansetmessages: + msg346271
2019-06-21 18:35:45ned.deilysetstatus: open -> closed

superseder: Infinite recursion tests triggering a segfault

nosy: + ned.deily
messages: + msg346242
resolution: duplicate
stage: resolved
2019-06-21 16:08:01Delgancreate