classification
Title: proxy_contains (weakref.proxy) can access an object with 0 refcount
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, colesbury, lukasz.langa, miss-islington, ned.deily, pablogsal
Priority: normal Keywords: patch

Created on 2019-10-07 14:49 by colesbury, last changed 2019-10-19 22:49 by benjamin.peterson. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16632 merged pablogsal, 2019-10-08 00:23
PR 16661 closed pablogsal, 2019-10-08 15:54
PR 16662 merged pablogsal, 2019-10-08 15:55
PR 16663 merged pablogsal, 2019-10-08 15:57
Messages (7)
msg354102 - (view) Author: Sam Gross (colesbury) Date: 2019-10-07 14:49
The implementation of weakref.proxy's methods call back into the Python API using a "borrowed reference" of the weakly referenced object (acquired via PyWeakref_GET_OBJECT). This API call may delete the last reference to the object (either directly or via GC), leaving a dangling pointer, which can be subsequently dereferenced.

Tested with Python 3.8.0b4 (debug build)

The following code crashes with a debug build of Python 3.8.0b4 on Linux.

import weakref

obj = None

class MyObj:
    def __iter__(self):
        global obj
        del obj
        return NotImplemented

obj = MyObj()
p = weakref.proxy(obj)
print(5 in p)


This particular test case does not crash with a release build (on Linux).

The implementation of `in` on a proxy object calls proxy_contains:

   return PySequence_Contains(PyWeakref_GET_OBJECT(proxy), value);

https://github.com/python/cpython/blob/v3.8.0b4/Objects/weakrefobject.c#L556


This eventually calls _PySequence_IterSearch. The call to PyObject_GetIter can call arbitrary code, which can lead to seq (the proxy's referent) being deleted. The subsequent call to type_error dereferences a dead object. 

    it = PyObject_GetIter(seq);
    if (it == NULL) {
        type_error("argument of type '%.200s' is not iterable", seq);
        return -1;
    }

https://github.com/python/cpython/blob/v3.8.0b4/Objects/abstract.c#L2003-L2007

I believe some functions, like proxy_length, may be immune to this problem because they do not access the borrowed referent after calling into user code. However, this is hard to verify from reading the code and may be fragile -- small changes to PyObject_Length/Size, for example, might .

See also https://bugs.python.org/issue16602
msg354157 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-10-08 00:38
I'm marking this as release blocker for the 3.8 incoming release so we don't forget about fixing this.
msg354161 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2019-10-08 02:05
Note I'm going to ignore this for the purposes of 2.7.17 because it doesn't look like a new regression.
msg354219 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-10-08 15:30
New changeset 10cd00a9e3c22af37c748ea5a417f6fb66601e21 by Pablo Galindo in branch 'master':
bpo-38395: Fix ownership in weakref.proxy methods (GH-16632)
https://github.com/python/cpython/commit/10cd00a9e3c22af37c748ea5a417f6fb66601e21
msg354231 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-10-08 20:38
New changeset 526ef856dd598fd3cefdfadeb18ede7a8e57aa41 by Pablo Galindo in branch '3.8':
[3.8] bpo-38395: Fix ownership in weakref.proxy methods (GH-16632) (GH-16662)
https://github.com/python/cpython/commit/526ef856dd598fd3cefdfadeb18ede7a8e57aa41
msg354490 - (view) Author: miss-islington (miss-islington) Date: 2019-10-11 20:29
New changeset 193366e25c4f84a58d2f6c6c577fd9f0143dc6e1 by Miss Islington (bot) (Pablo Galindo) in branch '3.7':
[3.7] bpo-38395: Fix ownership in weakref.proxy methods (GH-16632) (GH-16663)
https://github.com/python/cpython/commit/193366e25c4f84a58d2f6c6c577fd9f0143dc6e1
msg354978 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2019-10-19 22:49
All fixed now, right?
History
Date User Action Args
2019-10-19 22:49:35benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg354978

stage: patch review -> resolved
2019-10-11 20:29:08miss-islingtonsetnosy: + miss-islington
messages: + msg354490
2019-10-08 20:38:32pablogsalsetpriority: release blocker -> normal
2019-10-08 20:38:14pablogsalsetmessages: + msg354231
2019-10-08 15:57:33pablogsalsetpull_requests: + pull_request16246
2019-10-08 15:55:31pablogsalsetpull_requests: + pull_request16245
2019-10-08 15:54:27pablogsalsetpull_requests: + pull_request16244
2019-10-08 15:30:56pablogsalsetmessages: + msg354219
2019-10-08 02:05:29benjamin.petersonsetmessages: + msg354161
2019-10-08 00:38:56pablogsalsetnosy: + pablogsal
messages: + msg354157
2019-10-08 00:36:12pablogsalsetpriority: normal -> release blocker
nosy: + ned.deily, benjamin.peterson, lukasz.langa
2019-10-08 00:35:20pablogsalsetversions: + Python 2.7, Python 3.6
2019-10-08 00:26:53pablogsalsetversions: + Python 3.7, Python 3.9
2019-10-08 00:23:44pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16218
2019-10-07 14:49:09colesburycreate