Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash due to borrowed references in _PyStack_UnpackDict() #81088

Closed
jdemeyer opened this issue May 13, 2019 · 9 comments
Closed

Crash due to borrowed references in _PyStack_UnpackDict() #81088

jdemeyer opened this issue May 13, 2019 · 9 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@jdemeyer
Copy link
Contributor

BPO 36907
Nosy @vstinner, @encukou, @jdemeyer
PRs
  • bpo-36904: new function _PyStack_DictAsVector #13308
  • bpo-36907: fix refcount bug in _PyStack_UnpackDict() #13381
  • [3.7] bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) #13493
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-05-22.12:52:41.853>
    created_at = <Date 2019-05-13.19:40:03.967>
    labels = ['interpreter-core', '3.7', '3.8', 'type-crash']
    title = 'Crash due to borrowed references in _PyStack_UnpackDict()'
    updated_at = <Date 2019-05-22.12:52:41.852>
    user = 'https://github.com/jdemeyer'

    bugs.python.org fields:

    activity = <Date 2019-05-22.12:52:41.852>
    actor = 'petr.viktorin'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-05-22.12:52:41.853>
    closer = 'petr.viktorin'
    components = ['Interpreter Core']
    creation = <Date 2019-05-13.19:40:03.967>
    creator = 'jdemeyer'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36907
    keywords = ['patch']
    message_count = 9.0
    messages = ['342377', '342380', '342381', '343174', '343176', '343179', '343180', '343185', '343190']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'petr.viktorin', 'jdemeyer']
    pr_nums = ['13308', '13381', '13493']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue36907'
    versions = ['Python 3.7', 'Python 3.8']

    @jdemeyer
    Copy link
    Contributor Author

    class IntWithDict:
        def __init__(self, **kwargs):
            self.kwargs = kwargs
        def __index__(self):
            self.kwargs.clear()
            L = [2**i for i in range(10000)]
            return 0
    x = IntWithDict(dont_inherit=float())
    compile("", "", "", x, **x.kwargs)

    The above crashes CPython due to the usage of borrowed references in _PyStack_UnpackDict(): the dict x.kwargs contains the only reference to the float() object stored in x.kwargs

    When parsing the arguments, x.__int__() is called, which clears the dict, removing the only reference to that float()

    @jdemeyer jdemeyer added 3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump labels May 13, 2019
    @jdemeyer
    Copy link
    Contributor Author

    Ideally, this would be fixed together with bpo-36904.

    @jdemeyer
    Copy link
    Contributor Author

    The idea of bpo-36904 could be used here: define a special kind of tuple, which is like an ordinary tuple followed by a C array of PyObject* entries (all refcounted), terminated by a NULL to know where it ends. A special deallocation function would decref all entries.

    @encukou
    Copy link
    Member

    encukou commented May 22, 2019

    New changeset 77aa396 by Petr Viktorin (Jeroen Demeyer) in branch 'master':
    bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381)
    77aa396

    @encukou
    Copy link
    Member

    encukou commented May 22, 2019

    Jeroen, do you want to also do a backport for 3.7?

    @jdemeyer
    Copy link
    Contributor Author

    Jeroen, do you want to also do a backport for 3.7?

    Don't we have a bot for that?

    @encukou
    Copy link
    Member

    encukou commented May 22, 2019

    We do, but here the test will need to be changed:

    Python 3.7.3+ (heads/3.7:791e5fcbab, May 22 2019, 13:37:27) 
    [GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> class IntWithDict:
    ...     def __init__(self, **kwargs):
    ...         self.kwargs = kwargs
    ...     def __index__(self):
    ...         self.kwargs.clear()
    ...         return 0
    ... 
    >>> x = IntWithDict(dont_inherit=float())
    >>> compile("", "", "", x, **x.kwargs)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: an integer is required (got type IntWithDict)

    @jdemeyer
    Copy link
    Contributor Author

    Using __int__ instead of __index__ works. PR coming right away.

    @encukou
    Copy link
    Member

    encukou commented May 22, 2019

    New changeset d092caf by Petr Viktorin (Jeroen Demeyer) in branch '3.7':
    bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) (GH-13493)
    d092caf

    @encukou encukou closed this as completed May 22, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants