classification
Title: deepcopy of GenericAlias with __deepcopy__ method is broken
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: JacobHayes, gvanrossum, lukasz.langa, miss-islington, serhiy.storchaka, uriyyo
Priority: normal Keywords: patch

Created on 2021-09-10 17:09 by JacobHayes, last changed 2021-09-15 19:35 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28306 closed uriyyo, 2021-09-13 08:17
PR 28324 merged serhiy.storchaka, 2021-09-14 06:27
PR 28367 merged miss-islington, 2021-09-15 18:25
PR 28368 merged miss-islington, 2021-09-15 18:25
Messages (8)
msg401601 - (view) Author: Jacob Hayes (JacobHayes) Date: 2021-09-10 17:09
When deepcopying a parametrized types.GenericAlias (eg: a dict subclass) that has a __deepcopy__ method, the copy module doesn't detect the GenericAlias as a type and instead tries to call cls.__deepcopy__, passing `memo` inplace of self. This doesn't seem to happen with `typing.Generic` however.

Example:
```
from copy import deepcopy


class X(dict):
    def __deepcopy__(self, memo):
        return self


print(deepcopy(X()))
print(deepcopy(X))

print(type(X[str, int]))
print(deepcopy(X[str, int]()))
print(deepcopy(X[str, int]))
```
shows
```
{}
<class '__main__.X'>
<class 'types.GenericAlias'>
{}
Traceback (most recent call last):
  File "/tmp/demo.py", line 14, in <module>
    print(deepcopy(X[str, int]))
  File "/Users/jacobhayes/.pyenv/versions/3.9.6/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
TypeError: __deepcopy__() missing 1 required positional argument: 'memo'
```

I don't know if it's better to update `copy.deepcopy` here or perhaps narrow the `__getattr__` for `types.GenericAlias` (as `typing. _BaseGenericAlias` seems to).
msg401692 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-13 08:36
I think it is better to narrow the __getattr__. It can break protocols using other special methods.
msg401716 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-13 17:18
Serhiy, can you elaborate how GH-28306 can break protocols using other special methods? I can't come up with an example.
msg401736 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-13 21:23
I meant that the current __getattr__ breaks deepcopy(). It can break also other protocols. It already has a list of exceptions, we need to add "__copy__" and "__deepcopy__" in this list. Or maybe exclude all dunder names by default and use a white list?
msg401875 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-15 18:25
New changeset 5dce51a8875d9639786741e962b3cb208596b096 by Serhiy Storchaka in branch 'main':
bpo-45167: Fix deepcopying of GenericAlias (GH-28324)
https://github.com/python/cpython/commit/5dce51a8875d9639786741e962b3cb208596b096
msg401892 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-15 19:31
New changeset 2746045a37e0d56ffdd8bb79f961bd7df0d1afba by Miss Islington (bot) in branch '3.9':
bpo-45167: Fix deepcopying of GenericAlias (GH-28324) (GH-28368)
https://github.com/python/cpython/commit/2746045a37e0d56ffdd8bb79f961bd7df0d1afba
msg401897 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-15 19:35
New changeset de4c9c0bdc9b62e68dacd0c4bac20d7f4c527511 by Miss Islington (bot) in branch '3.10':
bpo-45167: Fix deepcopying of GenericAlias (GH-28324) (GH-28367)
https://github.com/python/cpython/commit/de4c9c0bdc9b62e68dacd0c4bac20d7f4c527511
msg401898 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-15 19:35
Thanks, Serhiy! ✨ 🍰 ✨
History
Date User Action Args
2021-09-15 19:35:55lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg401898

stage: patch review -> resolved
2021-09-15 19:35:20lukasz.langasetmessages: + msg401897
2021-09-15 19:31:23lukasz.langasetmessages: + msg401892
2021-09-15 18:25:54miss-islingtonsetpull_requests: + pull_request26782
2021-09-15 18:25:49miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26781
2021-09-15 18:25:48lukasz.langasetmessages: + msg401875
2021-09-14 06:27:04serhiy.storchakasetpull_requests: + pull_request26734
2021-09-13 21:23:03serhiy.storchakasetmessages: + msg401736
2021-09-13 17:18:12lukasz.langasetnosy: + lukasz.langa
messages: + msg401716
2021-09-13 08:36:45serhiy.storchakasetnosy: + serhiy.storchaka, gvanrossum
messages: + msg401692
2021-09-13 08:19:04uriyyosetversions: + Python 3.10, Python 3.11
2021-09-13 08:17:40uriyyosetkeywords: + patch
nosy: + uriyyo

pull_requests: + pull_request26720
stage: patch review
2021-09-10 17:09:52JacobHayescreate