classification
Title: Py_FatalError(): dump the list of extension modules
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: corona10, vstinner
Priority: normal Keywords: patch

Created on 2021-01-13 10:50 by vstinner, last changed 2021-01-21 09:49 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 24207 merged vstinner, 2021-01-13 11:22
PR 24240 merged vstinner, 2021-01-18 17:03
PR 24242 merged vstinner, 2021-01-18 17:07
PR 24254 merged vstinner, 2021-01-18 23:28
PR 24238 vstinner, 2021-01-18 23:33
Messages (11)
msg385013 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-13 10:50
When Python cannot report an error and its is not possible to recover from the error, Py_FatalError() displays an error message and exit immediately Python.

It's common that Python crashes are coming from third party extension modules, rather than Python itself. Recent example: bpo-42891 crash reported in Python, but it was a bug in lsm-db third party extension module.

I propose to enhance Py_FatalError() to attempt to dump the list of extension modules, to ease debugging.

I'm working on a PR.
msg385016 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-13 11:23
Example using PR 24207, the new part the "Extension modules:" list at the end:
---------------
$ ./python -X faulthandler
Python 3.10.0a4+ (heads/master-dirty:2396614b89, Jan 13 2021, 12:09:15) 
>>> import ctypes
>>> ctypes.string_at(0)
Fatal Python error: Segmentation fault

Current thread 0x00007f0aabd09740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in string_at
  File "<stdin>", line 1 in <module>

Extension modules:
 * sys
 * builtins
 * _imp
 * _thread
 * _warnings
 * _weakref
 * _io
 * marshal
 * posix
 * time
 * faulthandler
 * _codecs
 * _signal
 * _abc
 * _stat
 * readline
 * atexit
 * itertools
 * _operator
 * _collections
 * _functools
 * _opcode
 * _sre
 * _locale
 * _ctypes
 * _struct
---------------

Here you can see that the _ctypes module is loaded.
msg385020 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-13 11:28
> It's common that Python crashes are coming from third party extension modules, rather than Python itself. Recent example: bpo-42891 crash (...)

Another recent example: bpo-42879 "python: Objects/abstract.c:155: PyObject_GetItem: Assertion `(item != NULL) ^ (PyErr_Occurred() != NULL)' failed." was a bug in pygame.
msg385161 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-17 16:54
Example of a Pillow crash in Python 3.10:
https://mail.python.org/archives/list/python-dev@python.org/message/5DCR2R6POCOHTPFUOY4F5G7GTBYX6OPO/
msg385206 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-18 17:24
New changeset e232025025c8bd07c1d1b12a583a80c4a673f077 by Victor Stinner in branch 'master':
bpo-42923: Add Py_FatalError() test in test_capi (GH-24240)
https://github.com/python/cpython/commit/e232025025c8bd07c1d1b12a583a80c4a673f077
msg385207 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-18 17:35
New changeset 314b8787e0c50985ba708034b84ff5b37a1d47de by Victor Stinner in branch 'master':
bpo-42923: Py_FatalError() avoids fprintf() (GH-24242)
https://github.com/python/cpython/commit/314b8787e0c50985ba708034b84ff5b37a1d47de
msg385221 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-18 19:47
New changeset 250035d134ad482e724f73ce654682254b513ee0 by Victor Stinner in branch 'master':
bpo-42923: Dump extension modules on fatal error (GH-24207)
https://github.com/python/cpython/commit/250035d134ad482e724f73ce654682254b513ee0
msg385222 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-18 19:47
I created https://bugs.python.org/issue42955 to add sys.modules_names tuple: names of stdlib modules. Once it will be merged, I will updated this PR to filter the list of modules (ignore stdlib modules).
msg385303 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-19 22:35
New changeset 66f77caca39ba39ebe1e4a95dba6d19b20d51951 by Victor Stinner in branch 'master':
bpo-42923: _Py_DumpExtensionModules() ignores stdlib ext (GH-24254)
https://github.com/python/cpython/commit/66f77caca39ba39ebe1e4a95dba6d19b20d51951
msg385306 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-19 22:55
More "real world" example using cinder which imports 220 modules (ctypes is used to simulate a crash):
---------------
$ ./python -m venv env
$ env/bin/python -m pip install wheel
$ env/bin/python -m pip install cinder
$ env/bin/python -X dev -c 'import cinder, ctypes, sys; print(f"{len(sys.modules)=}"); print(); ctypes.string_at(0)'
(...)
len(sys.modules)=220

Fatal Python error: Segmentation fault

Current thread 0x00007fc4a88e0740 (most recent call first):
  File "/home/vstinner/python/master/Lib/ctypes/__init__.py", line 517 in string_at
  File "<string>", line 1 in <module>

Extension modules: greenlet._greenlet, __original_module__thread, __original_module_select, __original_module_time, _cffi_backend (total: 5)
Erreur de segmentation (core dumped)
---------------

cinder only uses 2 third party extension modules (on a total of 220 modules): greenlet and cffi.

Note: __original_xxx modules are aliases of stdlib modules created by eventlet monkey patching.

So if cinder does crash, I suggest to look at Python, but *also* look at these 2 extensions ;-)
msg385403 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-01-21 09:49
See also bpo-42955 "Add sys.module_names: list of stdlib module names (Python and extension modules)".
History
Date User Action Args
2021-01-21 09:49:26vstinnersetmessages: + msg385403
2021-01-19 22:55:01vstinnersetmessages: + msg385306
2021-01-19 22:36:24vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-01-19 22:35:34vstinnersetmessages: + msg385303
2021-01-18 23:33:40vstinnersetpull_requests: + pull_request23076
2021-01-18 23:28:02vstinnersetpull_requests: + pull_request23074
2021-01-18 19:47:59vstinnersetmessages: + msg385222
2021-01-18 19:47:21vstinnersetmessages: + msg385221
2021-01-18 17:35:04vstinnersetmessages: + msg385207
2021-01-18 17:24:52vstinnersetmessages: + msg385206
2021-01-18 17:07:48vstinnersetpull_requests: + pull_request23064
2021-01-18 17:03:34vstinnersetpull_requests: + pull_request23062
2021-01-17 16:54:42vstinnersetmessages: + msg385161
2021-01-14 00:44:32corona10setnosy: + corona10
2021-01-13 11:28:58vstinnersetmessages: + msg385020
2021-01-13 11:23:09vstinnersetmessages: + msg385016
2021-01-13 11:22:09vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request23034
2021-01-13 10:50:13vstinnercreate