Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash in ModuleType subclass with __slots__ #91047

Closed
ronaldoussoren opened this issue Mar 1, 2022 · 10 comments
Closed

Crash in ModuleType subclass with __slots__ #91047

ronaldoussoren opened this issue Mar 1, 2022 · 10 comments
Assignees
Labels
3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@ronaldoussoren
Copy link
Contributor

BPO 46891
Nosy @ronaldoussoren, @markshannon, @pablogsal, @sweeneyde
PRs
  • bpo-46891: Fix creating a new instance of a module subclass with slots #31643
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/markshannon'
    closed_at = <Date 2022-03-03.10:48:59.958>
    created_at = <Date 2022-03-01.21:25:06.644>
    labels = ['interpreter-core', 'release-blocker', 'type-crash', '3.11']
    title = 'Crash in ModuleType subclass with __slots__'
    updated_at = <Date 2022-03-03.10:48:59.958>
    user = 'https://github.com/ronaldoussoren'

    bugs.python.org fields:

    activity = <Date 2022-03-03.10:48:59.958>
    actor = 'Mark.Shannon'
    assignee = 'Mark.Shannon'
    closed = True
    closed_date = <Date 2022-03-03.10:48:59.958>
    closer = 'Mark.Shannon'
    components = ['Interpreter Core']
    creation = <Date 2022-03-01.21:25:06.644>
    creator = 'ronaldoussoren'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46891
    keywords = ['patch', '3.11regression']
    message_count = 10.0
    messages = ['414294', '414296', '414300', '414304', '414322', '414356', '414357', '414360', '414368', '414418']
    nosy_count = 4.0
    nosy_names = ['ronaldoussoren', 'Mark.Shannon', 'pablogsal', 'Dennis Sweeney']
    pr_nums = ['31643']
    priority = 'release blocker'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue46891'
    versions = ['Python 3.11']

    @ronaldoussoren
    Copy link
    Contributor Author

    The following crashes the interpreter in Python 3.11, and works fine in older versions:

    # --- script.py 
    import types
    
    class MyModule (types.ModuleType):
        __slots__ = (
            "_MyModule__a",
            "_MyModule__b",
        )
    
        def __init__(self, name):
            super().__init__(name)
    
    m = MyModule("name")
    # -- end of file

    The key in this is the __slots__ definition: The script does not crash without __slots__, or with a slots tuple with 1 item.

    This is a reproducer based on code in PyObjC.

    @ronaldoussoren ronaldoussoren added 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump labels Mar 1, 2022
    @ronaldoussoren
    Copy link
    Contributor Author

    This is with python 3.11 alpha 5 using the installer for macOS.

    @ronaldoussoren
    Copy link
    Contributor Author

    The crash itself happens during garbage collection:

    % python3.11 -Xdev t.py (master)pyobjc-8
    Fatal Python error: Segmentation fault

    Current thread 0x0000000104ce0580 (most recent call first):
    Garbage-collecting
    <no Python frame>

    And when using a debug build and -Xdev:

    ../Modules/gcmodule.c:2187: visit_validate: Assertion failed: PyObject_GC_Track() object is not valid
    Enable tracemalloc to get the memory block allocation traceback

    object address : 0x101a5f690
    object refcount : 1
    object type : 0x15503f230
    object type name: MyModule
    object repr : <module '?'>

    Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
    Python runtime state: initialized

    Current thread 0x00000001019c0580 (most recent call first):
    File "/Users/ronald/Projects/Forks/cpython/build/t.py", line 12 in <module>
    zsh: abort ./python.exe -Xdev t.py

    @ronaldoussoren
    Copy link
    Contributor Author

    The debug build seems to indicate that slot memory is not initiated properly, a stack trace (note the argument to visit_validate at frame #7):

    • frame #0: 0x000000019b60d9b8 libsystem_kernel.dylib__pthread_kill + 8 frame #1: 0x000000019b640eb0 libsystem_pthread.dylibpthread_kill + 288
      frame Rename README to README.rst and enhance formatting #2: 0x000000019b57e314 libsystem_c.dylibabort + 164 frame #3: 0x000000010030c4f4 python.exefatal_error_exit(status=-1) at pylifecycle.c:2624:9
      frame Update Python Software Foundation Copyright Year. #4: 0x000000010030c3cc python.exefatal_error(fd=2, header=1, prefix="_PyObject_AssertFailed", msg="_PyObject_AssertFailed", status=-1) at pylifecycle.c:2805:5 frame #5: 0x000000010030be20 python.exe_Py_FatalErrorFunc(func="_PyObject_AssertFailed", msg="_PyObject_AssertFailed") at pylifecycle.c:2821:5
      frame Add Pycharm's .idea directory to gitignore #6: 0x0000000100169fe0 python.exe_PyObject_AssertFailed(obj=0x0000000100d63690, expr=0x0000000000000000, msg="PyObject_GC_Track() object is not valid", file="../Modules/gcmodule.c", line=2187, function="visit_validate") at object.c:2293:5 frame #7: 0x0000000100351184 python.exevisit_validate(op=0xcdcdcdcdcdcdcdcd, parent_raw=0x0000000100d63690) at gcmodule.c:2186:9
      frame Change some mercurial/ hg.python.org references. #8: 0x000000010019845c python.exetraverse_slots(type=0x0000000101813230, self=0x0000000100d63690, visit=(python.exevisit_validate at gcmodule.c:2183), arg=0x0000000100d63690) at typeobject.c:1207:27
      frame Fix some deprecation warnings in Doc/conf.py #9: 0x0000000100197fe8 python.exesubtype_traverse(self=0x0000000100d63690, visit=(python.exevisit_validate at gcmodule.c:2183), arg=0x0000000100d63690) at typeobject.c:1228:23
      frame bpo-29474: Improve documentation for weakref.WeakValueDictionary #10: 0x0000000100350fdc python.exe`PyObject_GC_Track(op_raw=0x0000000100d63690) at gcmodule.c:2212:11

    @sweeneyde
    Copy link
    Member

    Bisected to here:

    c3f52b4 is the first bad commit
    commit c3f52b4
    Author: Mark Shannon <mark@hotpy.org>
    Date: Wed Jun 23 10:00:43 2021 +0100

    bpo-44486: Make sure that modules always have a dictionary. (GH-26847)
    
    * Make sure that modules always have a dictionary.
    

    @ronaldoussoren
    Copy link
    Contributor Author

    @pablo: this is a possible release blocker, hard crash for code that works in earlier releases.

    That said, I have a workaround on my end and IMHO a fix could wait for the next alpha.

    @pablogsal
    Copy link
    Member

    Marking this as release blocker.

    @markshannon
    Copy link
    Member

    Dennis, thanks for bisecting this.

    @ronaldoussoren
    Copy link
    Contributor Author

    I don't feel qualified to review the PR, but can confirm that this PR fixes this issue as well as the crash in PyObjC that I extracted my reproducer from.

    @markshannon
    Copy link
    Member

    New changeset 751c9ed by Mark Shannon in branch 'main':
    bpo-46891: Fix creating a new instance of a module subclass with slots (GH-31643)
    751c9ed

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants