classification
Title: "Fatal Python error: Cannot recover from stack overflow." from RecursionError in Python 3.5
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Richard Eames, iritkatriel, vstinner
Priority: normal Keywords:

Created on 2016-12-08 23:47 by Richard Eames, last changed 2020-11-25 00:05 by vstinner. This issue is now closed.

Messages (4)
msg282745 - (view) Author: Richard Eames (Richard Eames) Date: 2016-12-08 23:47
I've been porting a project to the latest version of Django, and due to one of the changes in the Django, caused a recursion error in my code. However, the error (under certain conditions) then causes the python interpreter to core dump. I'm not 100% sure what causes this to happen, but it does seem to be similar to https://bugs.python.org/issue6028

I've created a minimal django project: 
https://github.com/Naddiseo/python-core-dump

However, it does rely on some interaction between mysql, pymysql, and django to be reproduced, the latter two being 100% python code. I'm sorry that I could not reduce the test case further.

One of the interesting/weird things about this bug is that (on my machine at least) it requires exactly 15 entries in the `MIDDLEWARE` variable in "coredump/settings.py" in my test project, any more, or any less will cause the interpreter to issue a `RecursionError` as expected, but not to core dump. 

This appears to happen in 3.5, and not in 3.6 so perhaps whatever fix was applied to 3.6 can be backported to 3.5 so that it doesn't core dump?
msg381783 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-24 23:59
Hi Richard, 3.5 is no longer maintained. Does this issue exist in current (>= 3.8) versions?
msg381784 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-25 00:03
https://github.com/Naddiseo/python-core-dump reproducer no longer works on Python 3.10.

# in Python source code directory
git clone https://github.com/Naddiseo/python-core-dump
make
./python -m venv ENV
ENV/bin/python -m pip install -r python-core-dump/requirements.txt 
cd python-core-dump/
../ENV/bin/python manage.py migrate

Output:
---
/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/db/models/sql/query.py:11: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  from collections import Counter, Iterator, Mapping, OrderedDict
/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/core/paginator.py:101: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  class Page(collections.Sequence):
Traceback (most recent call last):
  File "/home/vstinner/python/master/python-core-dump/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/apps/config.py", line 199, in import_models
    self.models_module = import_module(models_module_name)
  File "/home/vstinner/python/master/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1037, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1014, in _find_and_load
  File "<frozen importlib._bootstrap>", line 993, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 687, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 831, in exec_module
  File "<frozen importlib._bootstrap>", line 235, in _call_with_frames_removed
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/contrib/auth/models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/vstinner/python/master/ENV/lib/python3.10/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
    class AbstractBaseUser(models.Model):
RuntimeError: __class__ not set defining 'AbstractBaseUser' as <class 'django.contrib.auth.base_user.AbstractBaseUser'>. Was __classcell__ propagated to type.__new__?
---
msg381785 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-25 00:05
> This appears to happen in 3.5, and not in 3.6 so perhaps whatever fix was applied to 3.6 can be backported to 3.5 so that it doesn't core dump?

I removed the stack usage for function calls in Python 3.6. See the "Stack consumption" section of:
https://vstinner.github.io/contrib-cpython-2017q1.html

> Hi Richard, 3.5 is no longer maintained. Does this issue exist in current (>= 3.8) versions?

He wrote that 3.6 is not affected. 3.5 no longer accept bugfixes, so I close the issue as out of date.
History
Date User Action Args
2020-11-25 00:05:52vstinnersetstatus: open -> closed
resolution: out of date
messages: + msg381785

stage: resolved
2020-11-25 00:03:43vstinnersetstatus: pending -> open
nosy: + vstinner
messages: + msg381784

2020-11-24 23:59:17iritkatrielsetstatus: open -> pending
nosy: + iritkatriel
messages: + msg381783

2016-12-08 23:48:33vstinnersettitle: "Fatal Python error: Cannot recover from stack overflow." from RecursionError -> "Fatal Python error: Cannot recover from stack overflow." from RecursionError in Python 3.5
2016-12-08 23:47:17Richard Eamescreate