classification
Title: Crash due to borrowed references in _PyStack_UnpackDict()
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jdemeyer, petr.viktorin, vstinner
Priority: normal Keywords: patch

Created on 2019-05-13 19:40 by jdemeyer, last changed 2019-05-22 12:52 by petr.viktorin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13308 closed jdemeyer, 2019-05-14 09:44
PR 13381 merged jdemeyer, 2019-05-17 10:35
PR 13493 merged jdemeyer, 2019-05-22 12:14
Messages (9)
msg342377 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-13 19:40
class IntWithDict:
    def __init__(self, **kwargs):
        self.kwargs = kwargs
    def __index__(self):
        self.kwargs.clear()
        L = [2**i for i in range(10000)]
        return 0
x = IntWithDict(dont_inherit=float())
compile("", "", "", x, **x.kwargs)


The above crashes CPython due to the usage of borrowed references in _PyStack_UnpackDict(): the dict x.kwargs contains the only reference to the float() object stored in x.kwargs

When parsing the arguments, x.__int__() is called, which clears the dict, removing the only reference to that float()
msg342380 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-13 19:51
Ideally, this would be fixed together with #36904.
msg342381 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-13 20:09
The idea of #36904 could be used here: define a special kind of tuple, which is like an ordinary tuple followed by a C array of PyObject* entries (all refcounted), terminated by a NULL to know where it ends. A special deallocation function would decref all entries.
msg343174 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2019-05-22 11:09
New changeset 77aa396bb9415428de09112ddf6b34bb843811eb by Petr Viktorin (Jeroen Demeyer) in branch 'master':
bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381)
https://github.com/python/cpython/commit/77aa396bb9415428de09112ddf6b34bb843811eb
msg343176 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2019-05-22 11:16
Jeroen, do you want to also do a backport for 3.7?
msg343179 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-22 11:35
> Jeroen, do you want to also do a backport for 3.7?

Don't we have a bot for that?
msg343180 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2019-05-22 11:41
We do, but here the test will need to be changed:

Python 3.7.3+ (heads/3.7:791e5fcbab, May 22 2019, 13:37:27) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class IntWithDict:
...     def __init__(self, **kwargs):
...         self.kwargs = kwargs
...     def __index__(self):
...         self.kwargs.clear()
...         return 0
... 
>>> x = IntWithDict(dont_inherit=float())
>>> compile("", "", "", x, **x.kwargs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type IntWithDict)
msg343185 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-22 12:12
Using __int__ instead of __index__ works. PR coming right away.
msg343190 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2019-05-22 12:52
New changeset d092caf096fa48baadfc0900792206bb5aa0192d by Petr Viktorin (Jeroen Demeyer) in branch '3.7':
bpo-36907: fix refcount bug in _PyStack_UnpackDict() (GH-13381) (GH-13493)
https://github.com/python/cpython/commit/d092caf096fa48baadfc0900792206bb5aa0192d
History
Date User Action Args
2019-05-22 12:52:41petr.viktorinsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-05-22 12:52:18petr.viktorinsetmessages: + msg343190
2019-05-22 12:14:06jdemeyersetpull_requests: + pull_request13408
2019-05-22 12:12:26jdemeyersetmessages: + msg343185
2019-05-22 11:41:43petr.viktorinsetmessages: + msg343180
2019-05-22 11:35:25jdemeyersetmessages: + msg343179
2019-05-22 11:16:59petr.viktorinsetmessages: + msg343176
2019-05-22 11:09:40petr.viktorinsetnosy: + petr.viktorin
messages: + msg343174
2019-05-17 10:35:05jdemeyersetpull_requests: + pull_request13292
2019-05-14 09:44:43jdemeyersetkeywords: + patch
stage: patch review
pull_requests: + pull_request13217
2019-05-13 20:09:21jdemeyersetmessages: + msg342381
2019-05-13 19:51:44jdemeyersetmessages: + msg342380
2019-05-13 19:40:22jdemeyersettype: crash
2019-05-13 19:40:03jdemeyercreate