Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python segfaults when configured with --with-pydebug --with-trace-refs #82581

Closed
pablogsal opened this issue Oct 7, 2019 · 9 comments
Closed
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@pablogsal
Copy link
Member

BPO 38400
Nosy @vstinner, @elprans, @pablogsal, @miss-islington
PRs
  • bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed #16630
  • [3.8] bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630) #18072
  • [3.7] bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630) #18073
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-10-07.23:43:22.405>
    created_at = <Date 2019-10-07.23:01:37.662>
    labels = ['interpreter-core', '3.9']
    title = 'Python segfaults when configured with --with-pydebug --with-trace-refs'
    updated_at = <Date 2020-01-19.23:51:01.644>
    user = 'https://github.com/pablogsal'

    bugs.python.org fields:

    activity = <Date 2020-01-19.23:51:01.644>
    actor = 'Elvis.Pranskevichus'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-10-07.23:43:22.405>
    closer = 'pablogsal'
    components = ['Interpreter Core']
    creation = <Date 2019-10-07.23:01:37.662>
    creator = 'pablogsal'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38400
    keywords = ['patch']
    message_count = 9.0
    messages = ['354151', '354152', '354153', '354171', '360279', '360280', '360281', '360282', '360283']
    nosy_count = 4.0
    nosy_names = ['vstinner', 'Elvis.Pranskevichus', 'pablogsal', 'miss-islington']
    pr_nums = ['16630', '18072', '18073']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue38400'
    versions = ['Python 3.9']

    @pablogsal
    Copy link
    Member Author

    configure --with-pydebug --with-trace-refs && make

    Modules/gcmodule.c:378: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
    <object at 0x5557629ca260 is freed>
    Fatal Python error: _PyObject_AssertFailed
    Python runtime state: preinitialized

    Current thread 0x00007f6e2863e740 (most recent call first):
    <no Python frame>
    /bin/sh: line 5: 28078 Aborted (core dumped) ./python -E -S -m sysconfig --generate-posix-vars
    generate-posix-vars failed
    make: * [Makefile:592: pybuilddir.txt] Error 1

    @pablogsal pablogsal added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Oct 7, 2019
    @pablogsal
    Copy link
    Member Author

    This is a regression introduced by:

    commit 6876257
    Author: Victor Stinner <vstinner@python.org>
    Date: Mon Oct 7 18:42:01 2019 +0200

    bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
    
    bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
    now also available in release mode. For example, it can be used to
    debug a crash in the visit_decref() function of the GC.
    
    Modify the following functions to also work in release mode:
    
    * _PyDict_CheckConsistency()
    * _PyObject_CheckConsistency()
    * _PyType_CheckConsistency()
    * _PyUnicode_CheckConsistency()
    
    Other changes:
    
    * _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
      (equals to 0).
    * _PyBytesWriter_CheckConsistency() now returns 1 and is only used
      with assert().
    * Reorder _PyObject_Dump() to write safe fields first, and only
      attempt to render repr() at the end.
    

    @pablogsal
    Copy link
    Member Author

    New changeset 36e33c3 by Pablo Galindo in branch 'master':
    bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630)
    36e33c3

    @vstinner
    Copy link
    Member

    vstinner commented Oct 8, 2019

    Notes for myself.

    the visit_decref() has been updated to display repr() of the parent object, rather than display the "freed" object.

    Before:
    ---
    Modules/gcmodule.c:378: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
    <object at 0x5557629ca260 is freed>
    Fatal Python error: _PyObject_AssertFailed
    Python runtime state: preinitialized

    Current thread 0x00007f6e2863e740 (most recent call first):
    <no Python frame>
    ---

    After:
    ---
    Modules/gcmodule.c:378: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
    Enable tracemalloc to get the memory block allocation traceback

    object address : 0x7fe2d1a52b40
    object refcount : 1
    object type : 0x740be0
    object type name: dict
    object repr : {'__repr__': <slot wrapper '__repr__' of 'weakref' objects>, '__hash__': <slot wrapper '__hash__' of 'weakref' objects>, '__call__': <slot wrapper '__call__' of 'weakref' objects>, '__lt__': <slot wrapper '__lt__' of 'weakref' objects>, '__le__': <slot wrapper '__le__' of 'weakref' objects>, '__eq__': <slot wrapper '__eq__' of 'weakref' objects>, '__ne__': <slot wrapper '__ne__' of 'weakref' objects>, '__gt__': <slot wrapper '__gt__' of 'weakref' objects>, '__ge__': <slot wrapper '__ge__' of 'weakref' objects>, '__init__': <slot wrapper '__init__' of 'weakref' objects>, '__new__': <built-in method __new__ of type object at 0x749500>, '__callback__': <member '__callback__' of 'weakref' objects>, '__doc__': None}

    Fatal Python error: _PyObject_AssertFailed
    Python runtime state: preinitialized

    Current thread 0x00007fe2dea35740 (most recent call first):
    <no Python frame>
    ---

    The problem was that the None object was seen as "freed" because _ob_prev=_ob_next=NULL. This object doesn't seem to be part of the list of all objects, sys.getobjects(). I fixed Py_None, but then I got the same assertion error on a different (moduledef) object. I gave up, and Pablo and me fixed _PyObject_IsFreed() instead.

    @elprans
    Copy link
    Mannequin

    elprans mannequin commented Jan 19, 2020

    Since https://bugs.python.org/issue38070 was backported to 3.8, shouldn't this as well? 3.8 is currently broken otherwise.

    @pablogsal
    Copy link
    Member Author

    Since https://bugs.python.org/issue38070 was backported to 3.8, shouldn't this as well? 3.8 is currently broken otherwise.

    Thanks for the catch, Elvis!

    @miss-islington
    Copy link
    Contributor

    New changeset a360070 by Miss Islington (bot) in branch '3.7':
    bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630)
    a360070

    @miss-islington
    Copy link
    Contributor

    New changeset 4cdb758 by Miss Islington (bot) in branch '3.8':
    bpo-38400 Don't check for NULL linked list pointers in _PyObject_IsFreed (GH-16630)
    4cdb758

    @elprans
    Copy link
    Mannequin

    elprans mannequin commented Jan 19, 2020

    Thanks for the lightning-fast turnaround, Pablo!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants