In bpo-30570, David Bolen noticed that "py -3.9 -m test test_pickle" consistently crashes on Windows (even though other methods of running that test do not crash, and the test succeeds when failed tests are retried).

Curiously, it seems that adding using support.infinite_recursion *introduced* the crash rather than preventing it.

I'm guessing this would have been fixed by #68689, but that fix was rolled back in #69366 for the sake of 3.9 ABI compatibility.

As of now, 3.9's pycore_ceval.c reads:

    static inline int _Py_RecursionLimitLowerWaterMark(int limit) {
        if (limit > 200) {
            return (limit - 50);
        }
        else {
            return (3 * (limit >> 2));
        }
    }

But support.infinite_recursion(max_depth=75) has a default 75, leaving a "low-water-mark" of 54, which the recursion apparently never recovers back to in the right way.

A couple of solutions could fix this:

(1) Remove the usage of support.infinite_recursion at the test.pickletester.AbstractPickleTests.test_bad_getattr call site.

(2) Use a different value max_depth. On my machine at least, it seems 75 was a "perfect storm" to cause this issue. Using infinite_recursion(60) or 61 or ... or 74 or 76 or 77 or 78 all pass, but infinite_recursion(75) in particular fails.

(3) Use fewer calls after the overflow by inlining something like assertRaises:

             with support.infinite_recursion():
-                self.assertRaises(RuntimeError, self.dumps, x, proto)
+                try:
+                    self.dumps(x, proto)
+                except RuntimeError:
+                    pass
+                else:
+                    self.fail("RuntimeError not raised")

(5) Re-visit an ABI-compliant version of #68689, such as #69347

The output I keep getting without any changes:

py -3.9 -m test test_pickle -v
...
test_attribute_name_interning (test.test_pickle.CPicklerTests) ... ok
test_bad_getattr (test.test_pickle.CPicklerTests) ... Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.Python runtime state: initialized

Current thread 0x000028b0 (most recent call first):
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
...
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 3300 in __getattr__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\copyreg.py", line 74 in _reduce_ex
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\test_pickle.py", line 65 in dumps
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 201 in handle
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 739 in assertRaises
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\pickletester.py", line 2381 in test_bad_getattr
File "C:\Users\sween\Source\Repos\cpython2\39\lib\test\support\init.py", line 1770 in wrapper
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 550 in _callTestMethod
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 592 in run
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\case.py", line 651 in __call__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 84 in __call__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 84 in __call__
File "C:\Users\sween\Source\Repos\cpython2\39\lib\unittest\suite.py", line 122 in run

I'll note that it also fails on first run on the Windows 11 builder:

https://buildbot.python.org/all/#/builders/737/builds/65

New changeset 4296396 by Mark Shannon in branch '3.9':
[3.9] bpo-45806: Fix recovery from stack overflow for 3.9. Again. (GH-29640)
4296396

Thanks! ✨ 🍰 ✨

Should this be backported to make the 3.8 Buildbots happy?