Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deepcopy of GenericAlias with __deepcopy__ method is broken #89330

Closed
JacobHayes mannequin opened this issue Sep 10, 2021 · 8 comments
Closed

deepcopy of GenericAlias with __deepcopy__ method is broken #89330

JacobHayes mannequin opened this issue Sep 10, 2021 · 8 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@JacobHayes
Copy link
Mannequin

JacobHayes mannequin commented Sep 10, 2021

BPO 45167
Nosy @gvanrossum, @ambv, @serhiy-storchaka, @miss-islington, @uriyyo, @JacobHayes
PRs
  • bpo-45167: Add ability to deepcopy types.GenericAlias #28306
  • bpo-45167: Fix deepcopying of GenericAlias #28324
  • [3.10] bpo-45167: Fix deepcopying of GenericAlias (GH-28324) #28367
  • [3.9] bpo-45167: Fix deepcopying of GenericAlias (GH-28324) #28368
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-09-15.19:35:55.718>
    created_at = <Date 2021-09-10.17:09:52.642>
    labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
    title = 'deepcopy of GenericAlias with __deepcopy__ method is broken'
    updated_at = <Date 2021-09-15.19:35:55.717>
    user = 'https://github.com/JacobHayes'

    bugs.python.org fields:

    activity = <Date 2021-09-15.19:35:55.717>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-09-15.19:35:55.718>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2021-09-10.17:09:52.642>
    creator = 'JacobHayes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45167
    keywords = ['patch']
    message_count = 8.0
    messages = ['401601', '401692', '401716', '401736', '401875', '401892', '401897', '401898']
    nosy_count = 6.0
    nosy_names = ['gvanrossum', 'lukasz.langa', 'serhiy.storchaka', 'miss-islington', 'uriyyo', 'JacobHayes']
    pr_nums = ['28306', '28324', '28367', '28368']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue45167'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @JacobHayes
    Copy link
    Mannequin Author

    JacobHayes mannequin commented Sep 10, 2021

    When deepcopying a parametrized types.GenericAlias (eg: a dict subclass) that has a deepcopy method, the copy module doesn't detect the GenericAlias as a type and instead tries to call cls.deepcopy, passing memo inplace of self. This doesn't seem to happen with typing.Generic however.

    Example:

    from copy import deepcopy
    
    
    class X(dict):
        def __deepcopy__(self, memo):
            return self
    
    
    print(deepcopy(X()))
    print(deepcopy(X))
    
    print(type(X[str, int]))
    print(deepcopy(X[str, int]()))
    print(deepcopy(X[str, int]))
    

    shows

    {}
    <class '__main__.X'>
    <class 'types.GenericAlias'>
    {}
    Traceback (most recent call last):
      File "/tmp/demo.py", line 14, in <module>
        print(deepcopy(X[str, int]))
      File "/Users/jacobhayes/.pyenv/versions/3.9.6/lib/python3.9/copy.py", line 153, in deepcopy
        y = copier(memo)
    TypeError: __deepcopy__() missing 1 required positional argument: 'memo'
    

    I don't know if it's better to update copy.deepcopy here or perhaps narrow the __getattr__ for types.GenericAlias (as typing. _BaseGenericAlias seems to).

    @JacobHayes JacobHayes mannequin added 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 10, 2021
    @uriyyo uriyyo added 3.10 only security fixes 3.11 only security fixes labels Sep 13, 2021
    @serhiy-storchaka
    Copy link
    Member

    I think it is better to narrow the __getattr__. It can break protocols using other special methods.

    @ambv
    Copy link
    Contributor

    ambv commented Sep 13, 2021

    Serhiy, can you elaborate how #72493 can break protocols using other special methods? I can't come up with an example.

    @serhiy-storchaka
    Copy link
    Member

    I meant that the current __getattr__ breaks deepcopy(). It can break also other protocols. It already has a list of exceptions, we need to add "__copy__" and "__deepcopy__" in this list. Or maybe exclude all dunder names by default and use a white list?

    @ambv
    Copy link
    Contributor

    ambv commented Sep 15, 2021

    New changeset 5dce51a by Serhiy Storchaka in branch 'main':
    bpo-45167: Fix deepcopying of GenericAlias (GH-28324)
    5dce51a

    @ambv
    Copy link
    Contributor

    ambv commented Sep 15, 2021

    New changeset 2746045 by Miss Islington (bot) in branch '3.9':
    bpo-45167: Fix deepcopying of GenericAlias (GH-28324) (GH-28368)
    2746045

    @ambv
    Copy link
    Contributor

    ambv commented Sep 15, 2021

    New changeset de4c9c0 by Miss Islington (bot) in branch '3.10':
    bpo-45167: Fix deepcopying of GenericAlias (GH-28324) (GH-28367)
    de4c9c0

    @ambv
    Copy link
    Contributor

    ambv commented Sep 15, 2021

    Thanks, Serhiy! ✨ 🍰 ✨

    @ambv ambv closed this as completed Sep 15, 2021
    @ambv ambv closed this as completed Sep 15, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants