Recently, after a small seemingly unrelated refactoring, the SymPy test suite in Python 3 started dying with "Fatal Python error: Cannot recover from stack overflow."  

Here's how to reproduce the error

git clone git://github.com/sympy/sympy.git # Clone the development version of SymPy
cd sympy
git checkout 0856119bd7399a416c21e1692855a1077164f21c # This is the first commit to exhibit the problem. Do this in case we make an unrelated change that removes the problem.
./bin/use2to3 # Convert the codebase to Python 3
python3 py3k-sympy/setup.py test # Run the tests

The issue is described in more detail at http://groups.google.com/group/sympy/browse_thread/thread/f664fe88e6b4f29d/3a44691c945695db#3a44691c945695db.  Some key points:

- The test that triggers the error is an XFAIL test (test that is expected to fail) that raises RuntimeError: maximum recursion depth exceeded.

- The change that caused the error, 0856119bd7399a416c21e1692855a1077164f21c (see https://github.com/sympy/sympy/commit/0856119bd7399a416c21e1692855a1077164f21c), is seemingly unrelated.  The only thing that I can think of is that it has added another call to the stack.

- This kills Python with Abort Trap: 6 in Mac OS X, which generates a problem report to be sent to Apple.  I have included a copy of it here: https://gist.github.com/2317869.

- Others have reproduced this error as well, as can be seen by our test reporter tool.  See the mailing list thread for more info.

- I tested this in 3.2.3r2, and the error still occurred.  I tried testing in the 3.3 alpha, but I could not get it to compile.

It looks like an issue in SymPy, a stack overflow. Why did you report this issue on CPython bug tracker?

Is there an infinite loop in a recursive function call?

Can you try to get the full Python traceback using faulthandler? Use "-X faulthandler" command line option, or use "import faulthandler; faulthandler.enable()" at the beginning of your program. You have to install it manually for Python < 3.3.

We do have a stack overflow, but this should be raising a RuntimeError, not killing Python.  The way it is now, Python dies completely with abort trap 6 (hence the Mac OS X problem report).  Sorry if I didn't make this clear in the OP.

SymPy uses a C extension, numpy. The problem may be related to numpy. CPython tries to catch segmentation fault, but when a C extension is used, CPython may fail to protect your program against stack overflow.

No it does not.  SymPy is a pure Python library.

The (first) Python stack overflow occurs at:
-----------
(gdb) py-bt
Traceback (most recent call first):
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2531, in expand
    from sympy.simplify.simplify import fraction
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper
    func_cache_it_cache[k] = r = func(*args, **kw_args)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag
    return (self.expand(deep, **hints), S.Zero)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex
    re_part, im_part = self.as_real_imag(deep=deep, **hints)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/expr.py", line 2551, in expand
    expr = func(deep=deep, **hints)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/core/cache.py", line 91, in wrapper
    func_cache_it_cache[k] = r = func(*args, **kw_args)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 514, in as_real_imag
    return (self.expand(deep, **hints), S.Zero)
  File "/home/haypo/issue14537/sympy/py3k-sympy/sympy/functions/elementary/hyperbolic.py", line 525, in _eval_expand_complex

(gdb) where
#0  _Py_CheckRecursiveCall (where=0x624f4b " in comparison") at Python/ceval.c:736
#1  0x0000000000419e6a in PyObject_RichCompare (v='sympy', w='sympy', op=2) at Objects/object.c:604
#2  0x0000000000419f2d in PyObject_RichCompareBool (v='sympy', w='sympy', op=2) at Objects/object.c:628
#3  0x00000000005e854e in lookdict (mp=0x7ffff1b3b1a8, key='sympy', hash=-191038920480525830) at Objects/dictobject.c:341
#4  0x00000000005e99ca in PyDict_GetItem (op=
    {'sympy.logic.inference': <module at remote 0x7fffeef80ba0>, ..., key='sympy') at Objects/dictobject.c:752
#5  0x00000000004f1510 in import_submodule (mod=None, subname='sympy', fullname='sympy') at Python/import.c:3313
#6  0x00000000004f064b in load_next (mod=None, altmod=None, inputname='sympy.simplify.simplify', p_outputname=0x7fffffe67e28, p_prefix=0x7fffffe67e20)
    at Python/import.c:3150
#7  0x00000000004ef610 in import_module_level (name='sympy.simplify.simplify', globals=
    {...}, locals=None, fromlist=('fraction',), level=0)
    at Python/import.c:2843
-----------

Ok, now I see the problem: import_submodule() uses PyDict_GetItem() whereas PyDict_GetItem() *clears* the exception (!). It should use PyDict_GetItemWithError() instead. Try attached patch. The problem is a generic problem: PyDict_GetItem() should be replaced by PyDict_GetItemWithError() everywhere... But it would be nice to start with import.c :-)