classification
Title: visit_decref(): add an assertion to check that the object is not freed
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, pablogsal, vstinner
Priority: normal Keywords: patch

Created on 2019-09-09 14:05 by vstinner, last changed 2019-10-16 01:37 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
bpo-38037-bug.patch vstinner, 2019-09-09 14:12
Pull Requests
URL Status Linked Edit
PR 15782 merged vstinner, 2019-09-09 14:09
PR 15793 merged miss-islington, 2019-09-09 15:45
PR 16243 merged vstinner, 2019-09-17 21:03
PR 16244 merged vstinner, 2019-09-17 21:10
PR 16245 merged vstinner, 2019-09-17 21:36
PR 16246 merged vstinner, 2019-09-17 22:01
PR 16258 merged vstinner, 2019-09-18 09:42
PR 16631 merged vstinner, 2019-10-07 23:53
PR 16816 closed vstinner, 2019-10-16 01:37
Messages (15)
msg351470 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-09 14:05
This issue is related to bpo-36389 "Add gc.enable_object_debugger(): detect corrupted Python objects in the GC.

I propose to call _PyObject_IsFreed() in visit_decref() in debug mode, to get a better error message if something goes wrong.

visit_decref() is commonly found in C traceback (ex: gdb backtrace) of bugs related to the garbage collector.
msg351473 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-09 14:12
> visit_decref() is commonly found in C traceback (ex: gdb backtrace) of bugs related to the garbage collector.

Example with attached bpo-38037-bug.patch: you can apply this patch on the master branch to reintroduce  bpo-38037 bug.

vstinner@apu$ git apply bpo-38037-bug.patch
vstinner@apu$ make

Without PR 15782:

vstinner@apu$ ./python -c pass 
Segmentation fault (core dumped)


With PR 15782:

vstinner@apu$ ./python -c pass 
Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
<object: freed>
Fatal Python error: _PyObject_AssertFailed

Current thread 0x00007f171a280740 (most recent call first):
Aborted (core dumped)


The bug is catched earlier. And Python provides more information.
msg351504 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-09 15:45
New changeset d91d4de31745fc1ed4c7e6c208917827c9c472b6 by Victor Stinner in branch 'master':
bpo-38070: visit_decref() calls _PyObject_IsFreed() (GH-15782)
https://github.com/python/cpython/commit/d91d4de31745fc1ed4c7e6c208917827c9c472b6
msg351537 - (view) Author: miss-islington (miss-islington) Date: 2019-09-09 17:18
New changeset 5731172bb1e958b1d80b18eaf88d3f2f93cfccdd by Miss Islington (bot) in branch '3.8':
bpo-38070: visit_decref() calls _PyObject_IsFreed() (GH-15782)
https://github.com/python/cpython/commit/5731172bb1e958b1d80b18eaf88d3f2f93cfccdd
msg351645 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-09-10 13:16
Should this be closed now?
msg351686 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-10 14:48
> Should this be closed now?

Please keep it open, I have a few more local changess to enhance Py_FatalError() :-) I would like to reuse the bpo number.
msg352665 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-17 21:36
New changeset 8fa3e1740b3f03ea65ddb68411c2238c5f98eec2 by Victor Stinner in branch 'master':
bpo-38070: _Py_DumpTraceback() writes <no Python frame> (GH-16244)
https://github.com/python/cpython/commit/8fa3e1740b3f03ea65ddb68411c2238c5f98eec2
msg352666 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-17 21:36
New changeset b39afb78768418d9405c4b528c80fa968ccc974d by Victor Stinner in branch 'master':
bpo-38070: Enhance _PyObject_Dump() (GH-16243)
https://github.com/python/cpython/commit/b39afb78768418d9405c4b528c80fa968ccc974d
msg352667 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-17 21:59
New changeset d3b904144e86e2442961de6a7dccecbe133d5c6d by Victor Stinner in branch 'master':
bpo-38070: Add _PyRuntimeState.preinitializing (GH-16245)
https://github.com/python/cpython/commit/d3b904144e86e2442961de6a7dccecbe133d5c6d
msg352675 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-17 23:35
New changeset 1ce16fb0977283ae42a9f8917bbca5f44aa69324 by Victor Stinner in branch 'master':
bpo-38070: Py_FatalError() logs runtime state (GH-16246)
https://github.com/python/cpython/commit/1ce16fb0977283ae42a9f8917bbca5f44aa69324
msg352676 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-17 23:38
msg351473 example with the new assertion and enhanced debug functions:
---
Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
<object at 0x7ff0038956d0 is freed>
Fatal Python error: _PyObject_AssertFailed
Python runtime state: finalizing (tstate=0x1d1c9b0)

Current thread 0x00007ff010841740 (most recent call first):
<no Python frame>
Aborted (core dumped)
---

* Obviously, "Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed" is the first most visible enhancement
* Add "Python runtime state: finalizing (tstate=0x1d1c9b0)": Python is finalizing (Py_Finalize)
* New "<no Python frame>" in the traceback
* "<Freed object>" became "<object at 0x7ff0038956d0 is freed>": add the object address
msg352720 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-18 12:10
New changeset 47bbab9f76735acc1991e541d12fd18be6b13b16 by Victor Stinner in branch '3.8':
[3.8] bpo-38070: Py_FatalError() logs runtime state (GH-16258)
https://github.com/python/cpython/commit/47bbab9f76735acc1991e541d12fd18be6b13b16
msg352721 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-18 12:10
Ok, I pushed the most important changes that I wanted to push. I close the issue.
msg354156 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-08 00:37
New changeset 4d5f94b8cd20f804c7868c5395a15aa6032f874c by Victor Stinner in branch 'master':
bpo-38070: Enhance visit_decref() debug trace (GH-16631)
https://github.com/python/cpython/commit/4d5f94b8cd20f804c7868c5395a15aa6032f874c
msg354681 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-15 01:06
New changeset f82ce5b1b12873b65927149a016be6a7c65e979d by Victor Stinner in branch '3.8':
[3.8] bpo-36389: Backport debug enhancements from master (GH-16796)
https://github.com/python/cpython/commit/f82ce5b1b12873b65927149a016be6a7c65e979d
History
Date User Action Args
2019-10-16 01:37:51vstinnersetpull_requests: + pull_request16368
2019-10-15 01:06:21vstinnersetmessages: + msg354681
2019-10-08 00:37:42vstinnersetmessages: + msg354156
2019-10-07 23:53:15vstinnersetpull_requests: + pull_request16217
2019-09-18 12:10:58vstinnersetstatus: open -> closed
versions: + Python 3.8
messages: + msg352721

resolution: fixed
stage: patch review -> resolved
2019-09-18 12:10:33vstinnersetmessages: + msg352720
2019-09-18 09:42:39vstinnersetpull_requests: + pull_request15853
2019-09-17 23:38:47vstinnersetmessages: + msg352676
2019-09-17 23:35:36vstinnersetmessages: + msg352675
2019-09-17 22:01:20vstinnersetpull_requests: + pull_request15842
2019-09-17 21:59:55vstinnersetmessages: + msg352667
2019-09-17 21:36:55vstinnersetpull_requests: + pull_request15841
2019-09-17 21:36:31vstinnersetmessages: + msg352666
2019-09-17 21:36:19vstinnersetmessages: + msg352665
2019-09-17 21:10:37vstinnersetpull_requests: + pull_request15840
2019-09-17 21:03:18vstinnersetpull_requests: + pull_request15839
2019-09-10 14:48:40vstinnersetmessages: + msg351686
2019-09-10 13:16:49pablogsalsetnosy: + pablogsal
messages: + msg351645
2019-09-09 17:18:12miss-islingtonsetnosy: + miss-islington
messages: + msg351537
2019-09-09 15:45:12miss-islingtonsetpull_requests: + pull_request15446
2019-09-09 15:45:02vstinnersetmessages: + msg351504
2019-09-09 14:12:09vstinnersetfiles: + bpo-38037-bug.patch

messages: + msg351473
2019-09-09 14:09:45vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request15435
2019-09-09 14:05:36vstinnercreate