user@development:~$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

def recurse(n):
    print(n)
    try:
        recurse(n+1)
    except RecursionError:
        print("recursion error")
    print(n)


Please note that there are 2 print(n) on the same level.

>>> recurse(0)
0
1
2
...
994
995
recursion error
994
993
...
2
1
0

Why is there no 2nd 995 after the recursion error?

Same happens for python 3.8.8 and 3.9.2.

Related question:

I also tried to set sys.setrecursionlimit(100000) and ran recurse(0), but at a bit beyond 21800, it just segfaulted.

Is there a way to determine the practically working maximum it can do?

> Why is there no 2nd 995 after the recursion error?

print(s) calls sys.stdout.write(s) and sys.stdout.write("\n"). The I/O stack consists of a text I/O wrapper, buffered writer, and a raw file object. What happens in this case is that writing "\n" causes the text wrapper to flush the buffer, which flushes to the raw file. Calling the latter's write() method raises a RecursionError. But "995\n" is still buffered and gets written out later on when the exception handler prints "recursion error".

Try replacing print() with low-level os.write(1, bytes_text). For example:

    import os

    def recurse(n):
        os.write(1, f'{n}\n'.encode())
        try:
            recurse(n + 1)
        except RecursionError:
            os.write(1, b'recursion error\n')
        os.write(1, f'{n}\n'.encode())

> Is there a way to determine the practically working maximum it can do?

The size of a C stack frame varies depending on the C function call, compiler optimizations, and the platform.

If you need to increase the recursion limit (not recommended) by a given factor, I suggest calling the recursive function in a new thread that has a stack size that's increased by the same factor over the default limit (depends on the platform). The stack size for new threads is set via old_size = threading.stack_size(new_size). Make sure that the already running threads (e.g. the main thread) that use the default stack size are executing code that stays well below the original recursion limit, else this could crash the interpreter.

In Windows, the default stack size is 2 MiB, and the maximum size that Python allows is one byte less than 256 MiB (268,435,455), which is about 128 times the default size. The largest size should safely (conservatively) support a recursion limit of 128,000 frames. With the above recurse() function, I tested that the maximum stack size crashes with a stack overflow at about 325,000 frames.

Eryk, thanks much for your detailled and clear explaining!

Can confirm that using os.write makes it raise the RecursionError where I expected it to be. Also print() raising the RecursionError explains the behaviour I have seen.

Sadly, this also shows that handling RecursionError is not as easy as one would wish it to be, because the usual place for the exception handler is still too close to it triggering (again) and every other usually harmless call could also trigger it on that level.

So maybe a better solution is voluntarily stopping recursion at a safe distance from the recursion limit (== not going deeper, avoiding the exception).

Or doing some tricky construction of first going upwards to a safe distance from the limit when handling this exception.