This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Descriptor resolution should own arguments passed to descriptors
Type: Stage: patch review
Components: C API Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dennis Sweeney, Mark.Shannon, rhettinger, tekknolagi
Priority: normal Keywords: patch

Created on 2022-01-28 07:06 by tekknolagi, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 30979 open tekknolagi, 2022-01-28 07:08
Messages (7)
msg411978 - (view) Author: Maxwell Bernstein (tekknolagi) * Date: 2022-01-28 07:06
Currently the descriptor (self) argument to __get__ is passed borrowed, since _PyType_LookupId returns a borrowed reference (see _PyObject_LookupSpecial and lookup_maybe_method in Objects/typeobject.c). This should instead own the reference.
msg411979 - (view) Author: Dennis Sweeney (Dennis Sweeney) * (Python committer) Date: 2022-01-28 07:25
Why? Callee-borrowing-from-caller is the established norm across the C API. You mention use-after-free, but can you elaborate on how that can happen in practice? says:

"""When you pass an object reference into another function, in general, the function borrows the reference from you — if it needs to store it, it will use Py_INCREF() to become an independent owner."""
msg412035 - (view) Author: Maxwell Bernstein (tekknolagi) * Date: 2022-01-28 22:05
Hi Dennis,

Sorry, let me be more clear. CPython in general ensures that objects passed in as arguments to a function will live for the duration of the function call if they are otherwise untouched. As it is now, this invariant is not maintained when calling the __get__ descriptor. Right now, it is not only borrowed by the callee but also not owned by the caller (!).

msg412040 - (view) Author: Maxwell Bernstein (tekknolagi) * Date: 2022-01-28 22:26
Ah, and another piece of the puzzle: this can happen in runtimes like Cinder that provide their own native code entrypoints to functions like a __get__.
msg412049 - (view) Author: Dennis Sweeney (Dennis Sweeney) * (Python committer) Date: 2022-01-29 03:38
Hi Max,

My apologies -- my first message was probably too dismissive.

Is there any way to make the existing code crash with pure Python, and can we then add such a test case? If not with pure Python, then perhaps with some minimal reproducible example using the C API?

Also, does the change have any effect on microbenchmarks involving the modified functions?
msg412250 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-02-01 11:57
# This needs to be C code for this to fail, I'm writing it in Python for clarity and brevity

class D:
    def __get__(self, instance, owner):
       del C.d
       # There are now no strong references to self
       self.whatever # Access to freed memory

# This can be Python

class C:
    d = D()

msg412277 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2022-02-01 15:21
If this can only be triggered from C code, it less of a concern.

The PR increases cost on a critical path, so we ought to be wary of applying it unless a known problem is being solved.

Note, this code path has survived two decades of deployment.  Also, the PR doesn't have a test to demonstrate that it fixes anything.  Those facts give some evidence that the PR has every user paying to solve a problem that almost no one actually has.
Date User Action Args
2022-04-11 14:59:55adminsetgithub: 90719
2022-02-01 15:21:47rhettingersetnosy: + rhettinger
messages: + msg412277
2022-02-01 11:57:47Mark.Shannonsetnosy: + Mark.Shannon
messages: + msg412250
2022-01-29 03:38:23Dennis Sweeneysetmessages: + msg412049
2022-01-28 22:26:38tekknolagisetmessages: + msg412040
2022-01-28 22:25:51tekknolagisetcomponents: + C API
2022-01-28 22:05:53tekknolagisetmessages: + msg412035
2022-01-28 07:25:57Dennis Sweeneysetnosy: + Dennis Sweeney
messages: + msg411979
2022-01-28 07:08:38tekknolagisetkeywords: + patch
stage: patch review
pull_requests: + pull_request29158
2022-01-28 07:06:14tekknolagicreate