Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

visit_decref(): add an assertion to check that the object is not freed #82251

Closed
vstinner opened this issue Sep 9, 2019 · 15 comments
Closed

visit_decref(): add an assertion to check that the object is not freed #82251

vstinner opened this issue Sep 9, 2019 · 15 comments
Labels
3.8 only security fixes 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@vstinner
Copy link
Member

vstinner commented Sep 9, 2019

BPO 38070
Nosy @vstinner, @pablogsal, @miss-islington
PRs
  • bpo-38070: visit_decref() calls _PyObject_IsFreed() #15782
  • [3.8] bpo-38070: visit_decref() calls _PyObject_IsFreed() (GH-15782) #15793
  • bpo-38070: Enhance _PyObject_Dump() #16243
  • bpo-38070: _Py_DumpTraceback() writes <no Python frame> #16244
  • bpo-38070: Add _PyRuntimeState.preinitializing #16245
  • bpo-38070: Py_FatalError() logs runtime state #16246
  • [3.8] bpo-38070: Py_FatalError() logs runtime state #16258
  • bpo-38070: Enhance visit_decref() debug trace #16631
  • [3.7] bpo-38070: visit_decref() calls _PyObject_IsFreed() in debug mode #16816
    		•
    Files
  • bpo-38037-bug.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-09-18.12:10:58.170>
created_at = <Date 2019-09-09.14:05:36.804>
labels = ['interpreter-core', 'type-feature', '3.8', '3.9']
title = 'visit_decref(): add an assertion to check that the object is not freed'
updated_at = <Date 2019-10-16.01:37:51.802>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2019-10-16.01:37:51.802>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-18.12:10:58.170>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2019-09-09.14:05:36.804>
creator = 'vstinner'
dependencies = []
files = ['48602']
hgrepos = []
issue_num = 38070
keywords = ['patch']
message_count = 15.0
messages = ['351470', '351473', '351504', '351537', '351645', '351686', '352665', '352666', '352667', '352675', '352676', '352720', '352721', '354156', '354681']
nosy_count = 3.0
nosy_names = ['vstinner', 'pablogsal', 'miss-islington']
pr_nums = ['15782', '15793', '16243', '16244', '16245', '16246', '16258', '16631', '16816']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38070'
versions = ['Python 3.8', 'Python 3.9']
    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 9, 2019

    This issue is related to bpo-36389 "Add gc.enable_object_debugger(): detect corrupted Python objects in the GC.

    I propose to call _PyObject_IsFreed() in visit_decref() in debug mode, to get a better error message if something goes wrong.

    visit_decref() is commonly found in C traceback (ex: gdb backtrace) of bugs related to the garbage collector.

    @vstinner vstinner added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Sep 9, 2019
    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 9, 2019

    visit_decref() is commonly found in C traceback (ex: gdb backtrace) of bugs related to the garbage collector.

    Example with attached bpo-38037-bug.patch: you can apply this patch on the master branch to reintroduce bpo-38037 bug.

    vstinner@apu$ git apply bpo-38037-bug.patch
    vstinner@apu$ make

    Without PR 15782:

    vstinner@apu$ ./python -c pass
    Segmentation fault (core dumped)

    With PR 15782:

    vstinner@apu$ ./python -c pass
    Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
    <object: freed>
    Fatal Python error: _PyObject_AssertFailed

    Current thread 0x00007f171a280740 (most recent call first):
    Aborted (core dumped)

    The bug is catched earlier. And Python provides more information.

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 9, 2019

    New changeset d91d4de by Victor Stinner in branch 'master':
    bpo-38070: visit_decref() calls _PyObject_IsFreed() (GH-15782)
    d91d4de

    @miss-islington
    Copy link
    Contributor

    New changeset 5731172 by Miss Islington (bot) in branch '3.8':
    bpo-38070: visit_decref() calls _PyObject_IsFreed() (GH-15782)
    5731172

    @pablogsal
    Copy link
    Member

    Should this be closed now?

    @vstinner
    Copy link
    Member Author

    Should this be closed now?

    Please keep it open, I have a few more local changess to enhance Py_FatalError() :-) I would like to reuse the bpo number.

    @vstinner
    Copy link
    Member Author

    New changeset 8fa3e17 by Victor Stinner in branch 'master':
    bpo-38070: _Py_DumpTraceback() writes <no Python frame> (GH-16244)
    8fa3e17

    @vstinner
    Copy link
    Member Author

    New changeset b39afb7 by Victor Stinner in branch 'master':
    bpo-38070: Enhance _PyObject_Dump() (GH-16243)
    b39afb7

    @vstinner
    Copy link
    Member Author

    New changeset d3b9041 by Victor Stinner in branch 'master':
    bpo-38070: Add _PyRuntimeState.preinitializing (GH-16245)
    d3b9041

    @vstinner
    Copy link
    Member Author

    New changeset 1ce16fb by Victor Stinner in branch 'master':
    bpo-38070: Py_FatalError() logs runtime state (GH-16246)
    1ce16fb

    @vstinner
    Copy link
    Member Author

    msg351473 example with the new assertion and enhanced debug functions:
    ---
    Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
    <object at 0x7ff0038956d0 is freed>
    Fatal Python error: _PyObject_AssertFailed
    Python runtime state: finalizing (tstate=0x1d1c9b0)

    Current thread 0x00007ff010841740 (most recent call first):
    <no Python frame>
    Aborted (core dumped)
    ---

    • Obviously, "Modules/gcmodule.c:379: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed" is the first most visible enhancement
    • Add "Python runtime state: finalizing (tstate=0x1d1c9b0)": Python is finalizing (Py_Finalize)
    • New "<no Python frame>" in the traceback
    • "<Freed object>" became "<object at 0x7ff0038956d0 is freed>": add the object address

    @vstinner
    Copy link
    Member Author

    New changeset 47bbab9 by Victor Stinner in branch '3.8':
    [3.8] bpo-38070: Py_FatalError() logs runtime state (GH-16258)
    47bbab9

    @vstinner
    Copy link
    Member Author

    Ok, I pushed the most important changes that I wanted to push. I close the issue.

    @vstinner vstinner added the 3.8 only security fixes label Sep 18, 2019
    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 8, 2019

    New changeset 4d5f94b by Victor Stinner in branch 'master':
    bpo-38070: Enhance visit_decref() debug trace (GH-16631)
    4d5f94b

    @vstinner
    Copy link
    Member Author

    New changeset f82ce5b by Victor Stinner in branch '3.8':
    [3.8] bpo-36389: Backport debug enhancements from master (GH-16796)
    f82ce5b

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @pablogsal @miss-islington