This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: weakref.proxy documentation might be outdated
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pablogsal Nosy List: aleneum, docs@python, jack__d, miss-islington, nascheme, pablogsal, pitrou, rhettinger
Priority: normal Keywords: patch

Created on 2021-06-28 14:15 by aleneum, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26936 closed jack__d, 2021-06-28 16:02
PR 26950 merged pablogsal, 2021-06-29 10:50
PR 26959 merged miss-islington, 2021-06-29 22:58
PR 26960 merged miss-islington, 2021-06-29 22:59
Messages (12)
msg396637 - (view) Author: Alexander Neumann (aleneum) Date: 2021-06-28 14:15
The documentation currently states:

> Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.

However, it seems with commit 96074de573f82fc66a2bd73c36905141a3f1d5c1 (https://github.com/python/cpython/commit/96074de573f82fc66a2bd73c36905141a3f1d5c1) hash/reversed pass throughs have been introduced that enable weakref.proxy to be used as a dictionary key. At least the following code fails with Python 3.8 but works with Python 3.9:

```
import weakref


class Model:
    pass


m = Model()
proxy = weakref.proxy(m)
ref = weakref.ref(m)
print(hash(m))
print(hash(ref))
print(hash(proxy))
```
msg396645 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-28 16:17
In my view, proxies should behave almost exactly as the underliying object, so using them is as ergonomic as possible. For instance, when using them as a way to avoid cycles, is quite annoying if you have a hashable object in a dictionary and they fail to tell you that is there:

>>> class A:
...   ...
...
>>> a = A()
>>> d = {a: None}
>>> weakref.proxy(a) in d
True

but doing this in 3.8 is impossible:

>>> import weakref
>>> class A:
...    ...
...
>>> a = A()
>>> d = {a: None}
>>> weakref.proxy(a) in d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'weakproxy'

Which defies a lot of use cases of this. Notice that if the object is not hashable because the author of the class wants to prevent against that it will work as expected:

>>> class A:
...    __hash__ = None
...
>>> a = A()
>>> hash(weakref.proxy(a))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'A'

This also gives a better error message because it explains that A is not hashable.
msg396647 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-28 16:24
I assume that the original comment is referring to the case when a proxy is a key and the original dies, but this just makes the thing inoperable, as expected:

>>> class A:
...   ...
...
>>> a = A()
>>> p = weakref.proxy(a)
>>> d = {p: 42}
>>> a in d
True
>>> del a
>>> 4 in d
False
>>> None in d
False
>>> x in d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ReferenceError: weakly-referenced object no longer exists

So any operation on the proxy will raise (because is death) but other operations on the dict will work nicely. For hashable objects, it explicitly breaks the hash because they stop being hashable:

>>> class A:
...   ...
...
>>> a = A()
>>> p = weakref.proxy(a)
>>> t = (1,2,p)
>>> hash(t)
3027598395945759125
>>> del a
>>> hash(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ReferenceError: weakly-referenced object no longer exists

I don't think this is a problem and that therefore we should make the proxy not forward the hash, but maybe there is some opposing opinions to this, so let's give some time for other core devss to comment in case they have other views/concerns
msg396648 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-28 16:28
Antoine, Neil, what do you think about this?
msg396662 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-06-28 20:27
Pablo, I'm having second thoughts on hash forwarding.  It looks like its omission was an intentional design decision.  I vote for restoring the previous behavior.  That will likely save some headaches for some users.
msg396665 - (view) Author: Jack DeVries (jack__d) * Date: 2021-06-28 21:47
I'm going to close my PR; it seems like documentation is not the direction we're going, so no need to keep it open.
msg396747 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2021-06-29 17:16
It seems to me the old behaviour (don't forward hash) was done for good reasons.  If the referent might go away, it is not valid to use it as a dict key since the hash and eq result changes.  If it can't go away, what reason is there to use a weakref and not a direct reference?
msg396749 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-29 17:30
Thanks Neil for your input. I am indeed reverting that change to its previous behaviour.
msg396750 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-29 17:31
Just also to be clear on my original message, reverting it to a previous behaviour would disallow some patterns that could be useful:

>>> class A:
...   ...
...
>>> a = A()
>>> d = {a: None}
>>> weakref.proxy(a) in d
True

But I agree is better to revert back in any case
msg396761 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-29 22:58
New changeset e2fea101fd5517f33371b04432842b971021c3bf by Pablo Galindo in branch 'main':
bpo-44523: Remove the pass-through for hash() in weakref proxy objects (GH-26950)
https://github.com/python/cpython/commit/e2fea101fd5517f33371b04432842b971021c3bf
msg396762 - (view) Author: miss-islington (miss-islington) Date: 2021-06-29 23:19
New changeset 2df13e1211cf420bf6557df02e694bf1653a0ebe by Miss Islington (bot) in branch '3.10':
bpo-44523: Remove the pass-through for hash() in weakref proxy objects (GH-26950)
https://github.com/python/cpython/commit/2df13e1211cf420bf6557df02e694bf1653a0ebe
msg396763 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-29 23:19
New changeset f790bc8084d3dfd723889740f9129ac8fcb2fa02 by Miss Islington (bot) in branch '3.9':
bpo-44523: Remove the pass-through for hash() in weakref proxy objects (GH-26950) (GH-26960)
https://github.com/python/cpython/commit/f790bc8084d3dfd723889740f9129ac8fcb2fa02
History
Date User Action Args
2022-04-11 14:59:47adminsetgithub: 88689
2021-06-29 23:19:36pablogsalsetmessages: + msg396763
2021-06-29 23:19:31miss-islingtonsetmessages: + msg396762
2021-06-29 23:08:22pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-06-29 22:59:01miss-islingtonsetpull_requests: + pull_request25526
2021-06-29 22:58:57pablogsalsetmessages: + msg396761
2021-06-29 22:58:55miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25525
2021-06-29 17:31:47pablogsalsetmessages: + msg396750
2021-06-29 17:30:45pablogsalsetmessages: + msg396749
2021-06-29 17:16:08naschemesetmessages: + msg396747
2021-06-29 10:50:38pablogsalsetpull_requests: + pull_request25518
2021-06-28 21:47:54jack__dsetmessages: + msg396665
2021-06-28 20:27:52rhettingersetnosy: + rhettinger
messages: + msg396662
2021-06-28 16:28:53pablogsalsetnosy: + nascheme, pitrou
messages: + msg396648
2021-06-28 16:24:46pablogsalsetmessages: + msg396647
2021-06-28 16:17:27pablogsalsetmessages: + msg396645
2021-06-28 16:02:37jack__dsetkeywords: + patch
nosy: + jack__d

pull_requests: + pull_request25505
stage: patch review
2021-06-28 15:17:43rhettingersetassignee: docs@python -> pablogsal

nosy: + pablogsal
2021-06-28 14:15:05aleneumcreate