This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: help("modules") segfaults on 3.11, MacOS
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.11
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, christian.heimes, erlendaasland, iritkatriel
Priority: normal Keywords: 3.11regression

Created on 2021-11-30 22:45 by iritkatriel, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (16)
msg407403 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 22:45
Python 3.11.0a2+ (heads/main:8a45ca542a, Nov 30 2021, 22:40:56) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> help("modules")

Please wait a moment while I gather a list of all available modules...

/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  __import__(info.name)
/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: lib2to3 package is deprecated and may not be able to parse Python 3.10+
  __import__(info.name)
test_sqlite3: testing with version '2.6.0', sqlite_version '3.32.3'
/Users/iritkatriel/src/cpython/Lib/distutils/command/build_ext.py:13: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  from distutils.sysconfig import customize_compiler, get_python_version
zsh: segmentation fault  ./python.exe
msg407404 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 22:46
On the debugger: 

>>> help("modules")

Please wait a moment while I gather a list of all available modules...

/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  __import__(info.name)
/Users/iritkatriel/src/cpython/Lib/pkgutil.py:92: DeprecationWarning: lib2to3 package is deprecated and may not be able to parse Python 3.10+
  __import__(info.name)
test_sqlite3: testing with version '2.6.0', sqlite_version '3.32.3'
/Users/iritkatriel/src/cpython/Lib/distutils/command/build_ext.py:13: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  from distutils.sysconfig import customize_compiler, get_python_version
Process 88596 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000100160046 python.exe`_PyObject_GenericGetAttrWithDict(obj=0x0000000106acfa50, name=0x0000000100fe91c0, dict=0x0000000000000000, suppress=0) at object.c:1305:17
   1302	        PyDictValues **values_ptr = _PyObject_ValuesPointer(obj);
   1303	        if (values_ptr && *values_ptr) {
   1304	            if (PyUnicode_CheckExact(name)) {
-> 1305	                assert(*_PyObject_DictPointer(obj) == NULL);
   1306	                res = _PyObject_GetInstanceAttribute(obj, *values_ptr, name);
   1307	                if (res != NULL) {
   1308	                    goto done;
Target 0: (python.exe) stopped.
(lldb) p _PyObject_Dump(name)
object address  : 0x100fe91c0
object refcount : 165
object type     : 0x10051aec0
object type name: str
object repr     : 'string'
(lldb) p _PyObject_Dump(*_PyObject_DictPointer(obj))
error: Execution was interrupted, reason: Attempted to dereference an invalid pointer..
The process has been returned to the state before expression evaluation.
(lldb) p _PyObject_Dump(obj)
object address  : 0x106acfa50
object refcount : 7
object type     : 0x105add590
object type name: _cffi_backend.FFI
object repr     : <_cffi_backend.FFI object at 0x106acfa50>
msg407405 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 22:49
It works for me on 3.10.
msg407406 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-30 23:13
It's failing in this assertion which was added in https://github.com/python/cpython/pull/28802 :

https://github.com/python/cpython/blob/8a45ca542a65ea27e7acaa44a4c833a27830e796/Objects/object.c#L1305
msg407442 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-12-01 11:45
Works for me on Ubuntu.

Python 3.11.0a2+ (heads/main:0aa0bd0563, Dec  1 2021, 11:39:40) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help("modules")

[lots of output, but no assertion failure or segfault]

>>> 


I note that your debug output includes `_cffi_backend.FFI object`
Is this a cffi bug?
msg407443 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-12-01 11:47
When I say "cffi bug", I should say "is cffi not ported to 3.11 yet?"
msg407447 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-12-01 12:21
Unable to reproduce on macOS 12 with latest 3.11 official build, and my most recent dev build (HEAD at b394af13f69c1a2ac0e7d32dabf6d86443606ab2).
msg407450 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-12-01 12:27
Also unable to reproduce with HEAD at 49444fb807ecb396462c8e5f547eeb5c6bc5d4de.
msg407451 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-01 12:29
I'm using MacOS 11.6.
msg407452 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-12-01 12:31
Also unable to reproduce with HEAD at 8a45ca542a65ea27e7acaa44a4c833a27830e796. Perhaps it is related to your build/dev environment?
msg407455 - (view) Author: Erlend E. Aasland (erlendaasland) * (Python triager) Date: 2021-12-01 13:23
I'm unable to reproduce on macOS 11.6 as well (head at 49444fb807ecb396462c8e5f547eeb5c6bc5d4de).
msg407477 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-01 18:52
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x000000010015fd26 python.exe`_PyObject_GenericGetAttrWithDict(obj=0x000000013feb7a50, name=0x0000000100fe9350, dict=0x0000000000000000, suppress=0) at object.c:1305:17
   1302	        PyDictValues **values_ptr = _PyObject_ValuesPointer(obj);
   1303	        if (values_ptr && *values_ptr) {
   1304	            if (PyUnicode_CheckExact(name)) {
-> 1305	                assert(*_PyObject_DictPointer(obj) == NULL);
   1306	                res = _PyObject_GetInstanceAttribute(obj, *values_ptr, name);
   1307	                if (res != NULL) {
   1308	                    goto done;
Target 0: (python.exe) stopped.
(lldb) p obj
(PyObject *) $0 = 0x000000013feb7a50
(lldb) p _PyObject_Dump((PyObject*)obj->ob_type)
object address  : 0x13ff25590
object refcount : 32
object type     : 0x100518680
object type name: type
object repr     : <class '_cffi_backend.FFI'>
(lldb) p (PyObject*)obj->ob_type
(PyObject *) $1 = 0x000000013ff25590
(lldb) p (PyObject*)obj->ob_type->tp_dict
(PyObject *) $2 = 0x000000013fedd910
(lldb) p _PyObject_Dump((PyObject*)obj->ob_type->tp_dict)
object address  : 0x13fedd910
object refcount : 1
object type     : 0x100511d50
object type name: dict
object repr     : {'__new__': <built-in method __new__ of type object at 0x13ff25590>, '__getattribute__': <slot wrapper '__getattribute__' of '_cffi_backend.FFI' objects>, '__init__': <slot wrapper '__init__' of '_cffi_backend.FFI' objects>, 'addressof': <method 'addressof' of '_cffi_backend.FFI' objects>, 'alignof': <method 'alignof' of '_cffi_backend.FFI' objects>, 'def_extern': <method 'def_extern' of '_cffi_backend.FFI' objects>, 'callback': <method 'callback' of '_cffi_backend.FFI' objects>, 'cast': <method 'cast' of '_cffi_backend.FFI' objects>, 'dlclose': <method 'dlclose' of '_cffi_backend.FFI' objects>, 'dlopen': <method 'dlopen' of '_cffi_backend.FFI' objects>, 'from_buffer': <method 'from_buffer' of '_cffi_backend.FFI' objects>, 'from_handle': <method 'from_handle' of '_cffi_backend.FFI' objects>, 'gc': <method 'gc' of '_cffi_backend.FFI' objects>, 'getctype': <method 'getctype' of '_cffi_backend.FFI' objects>, 'init_once': <method 'init_once' of '_cffi_backend.FFI' objects>, 'integer_const': <method 'integer_const' of '_cffi_backend.FFI' objects>, 'list_types': <method 'list_types' of '_cffi_backend.FFI' objects>, 'memmove': <method 'memmove' of '_cffi_backend.FFI' objects>, 'new': <method 'new' of '_cffi_backend.FFI' objects>, 'new_allocator': <method 'new_allocator' of '_cffi_backend.FFI' objects>, 'new_handle': <method 'new_handle' of '_cffi_backend.FFI' objects>, 'offsetof': <method 'offsetof' of '_cffi_backend.FFI' objects>, 'release': <method 'release' of '_cffi_backend.FFI' objects>, 'sizeof': <method 'sizeof' of '_cffi_backend.FFI' objects>, 'string': <method 'string' of '_cffi_backend.FFI' objects>, 'typeof': <method 'typeof' of '_cffi_backend.FFI' objects>, 'unpack': <method 'unpack' of '_cffi_backend.FFI' objects>, 'errno': <attribute 'errno' of '_cffi_backend.FFI' objects>, '__doc__': None, 'NULL': <cdata 'void *' NULL>, 'error': <class 'ffi.error'>, 'CType': <class '_cffi_backend.CType'>, 'CData': <class '_cffi_backend._CDataBase'>, 'buffer': <class '_cffi_backend.buffer'>, 'RTLD_LAZY': 1, 'RTLD_NOW': 2, 'RTLD_GLOBAL': 8, 'RTLD_LOCAL': 4, 'RTLD_NODELETE': 128, 'RTLD_NOLOAD': 16}
(lldb) p (PyObject*)obj->ob_type->tp_dictoffset
(PyObject *) $3 = NULL



Because the tp_dictoffset is 0, _PyObject_DictPointer(obj) returns NULL and in the assertion, where the return value is dereferenced, it crashes.

(Note: it's not that the assertion is failing, we don't get that far).
msg407479 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-01 18:54
This looks like an issue with 3rd party software cffi. Are you using a fresh build of cffi? The internal ABI of an alpha build changes very often.
msg407480 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-01 18:58
How do I check that?
msg407481 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-12-01 19:01
The easiest way is to remove and rebuild 3rd party software.
msg407495 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-01 21:57
Christian - that's what it was. I uninstalled cffi and now it works.
History
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90099
2021-12-01 21:57:56iritkatrielsetstatus: open -> closed
resolution: not a bug
messages: + msg407495

stage: resolved
2021-12-01 19:01:50christian.heimessetmessages: + msg407481
2021-12-01 18:58:33iritkatrielsetmessages: + msg407480
2021-12-01 18:54:48christian.heimessetnosy: + christian.heimes
messages: + msg407479
2021-12-01 18:52:19iritkatrielsetmessages: + msg407477
2021-12-01 13:23:31erlendaaslandsetmessages: + msg407455
2021-12-01 12:31:08erlendaaslandsetmessages: + msg407452
2021-12-01 12:29:31iritkatrielsetmessages: + msg407451
2021-12-01 12:27:07erlendaaslandsetmessages: + msg407450
2021-12-01 12:21:41erlendaaslandsetnosy: + erlendaasland
messages: + msg407447
2021-12-01 11:47:36Mark.Shannonsetmessages: + msg407443
2021-12-01 11:45:51Mark.Shannonsetmessages: + msg407442
2021-11-30 23:13:07iritkatrielsetnosy: + Mark.Shannon
messages: + msg407406
2021-11-30 22:49:31iritkatrielsetkeywords: + 3.11regression

messages: + msg407405
2021-11-30 22:47:29iritkatrielsettitle: help("modules") segfaults on 3.11 -> help("modules") segfaults on 3.11, MacOS
2021-11-30 22:46:12iritkatrielsetmessages: + msg407404
2021-11-30 22:45:55iritkatrielcreate