Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dataclasses that inherit from Protocol subclasses have wrong __init__ #89244

Closed
julianfortune mannequin opened this issue Sep 1, 2021 · 7 comments
Closed

dataclasses that inherit from Protocol subclasses have wrong __init__ #89244

julianfortune mannequin opened this issue Sep 1, 2021 · 7 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@julianfortune
Copy link
Mannequin

julianfortune mannequin commented Sep 1, 2021

BPO 45081
Nosy @ericvsmith, @ambv, @serhiy-storchaka, @icgood, @miss-islington, @uriyyo, @Fidget-Spinner, @julianfortune, @edgarrmondragon, @shr1k
PRs
  • bpo-45081: Fix __init__ method generation when inheriting from Protocol #28121
  • [3.10] bpo-45081: Fix __init__ method generation when inheriting from Protocol (GH-28121) #28131
  • [3.9] bpo-45081: Fix __init__ method generation when inheriting from Protocol (GH-28121) #28132
  • Files
  • test.py: Example of another issue with Protocol subclassing
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-09-02.18:46:32.940>
    created_at = <Date 2021-09-01.21:06:11.095>
    labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
    title = 'dataclasses that inherit from Protocol subclasses have wrong __init__'
    updated_at = <Date 2021-09-12.21:38:03.668>
    user = 'https://github.com/julianfortune'

    bugs.python.org fields:

    activity = <Date 2021-09-12.21:38:03.668>
    actor = 'icgood'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-09-02.18:46:32.940>
    closer = 'julianfortune'
    components = ['Library (Lib)']
    creation = <Date 2021-09-01.21:06:11.095>
    creator = 'julianfortune'
    dependencies = []
    files = ['50277']
    hgrepos = []
    issue_num = 45081
    keywords = ['patch']
    message_count = 7.0
    messages = ['400868', '400936', '400939', '400979', '401647', '401671', '401672']
    nosy_count = 10.0
    nosy_names = ['eric.smith', 'lukasz.langa', 'serhiy.storchaka', 'icgood', 'miss-islington', 'uriyyo', 'kj', 'julianfortune', 'edgarrmondragon', 'shrik']
    pr_nums = ['28121', '28131', '28132']
    priority = 'normal'
    resolution = None
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue45081'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @julianfortune
    Copy link
    Mannequin Author

    julianfortune mannequin commented Sep 1, 2021

    I believe bpo-44806: Fix __init__ in subclasses of protocols has caused a regression when using a Dataclass.

    In Python 3.9.7, a dataclass that inherits from a subclass of typing.Protocol (i.e., a user-defined protocol), does not have the correct __init__.

    ### Demonstration

    from dataclasses import dataclass
    from typing import Protocol
    
    class P(Protocol):
        pass
    
    @dataclass
    class B(P):
        value: str
    
    print(B("test"))

    In 3.9.7:

    Traceback (most recent call last):
      File "test.py", line 11, in <module>
        print(B("test"))
    TypeError: B() takes no arguments

    In 3.9.6:

    B(value='test')

    ### Affected Projects

    @julianfortune julianfortune mannequin added 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 1, 2021
    @ambv
    Copy link
    Contributor

    ambv commented Sep 2, 2021

    New changeset 0635e20 by Yurii Karabas in branch 'main':
    bpo-45081: Fix __init__ method generation when inheriting from Protocol (GH-28121)
    0635e20

    @ambv
    Copy link
    Contributor

    ambv commented Sep 2, 2021

    New changeset 98eb408 by Miss Islington (bot) in branch '3.9':
    bpo-45081: Fix __init__ method generation when inheriting from Protocol (GH-28121) (GH-28132)
    98eb408

    @julianfortune julianfortune mannequin closed this as completed Sep 2, 2021
    @julianfortune julianfortune mannequin closed this as completed Sep 2, 2021
    @miss-islington
    Copy link
    Contributor

    New changeset 79e9f5a by Miss Islington (bot) in branch '3.10':
    bpo-45081: Fix __init__ method generation when inheriting from Protocol (GH-28121)
    79e9f5a

    @ambv ambv added 3.10 only security fixes 3.11 only security fixes labels Sep 3, 2021
    @icgood
    Copy link
    Mannequin

    icgood mannequin commented Sep 11, 2021

    I believe this was a deeper issue that affected all classes inheriting Protocol, causing a TypeError on even the most basic case (see attached):

    Traceback (most recent call last):
      File "/.../test.py", line 14, in <module>
        MyClass()
      File "/.../test.py", line 11, in __init__
        super().__init__()
      File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 1083, in _no_init
        raise TypeError('Protocols cannot be instantiated')
    TypeError: Protocols cannot be instantiated

    This was a new regression in 3.9.7 and seems to be resolved by this fix. The desired behavior should be supported according to PEP-544: https://www.python.org/dev/peps/pep-0544/#explicitly-declaring-implementation

    @julianfortune
    Copy link
    Mannequin Author

    julianfortune mannequin commented Sep 12, 2021

    Ian,

    MyProtocol does not provide an __init__(), and thus

    super().__init__()
    

    is calling the __init__() from Protocol. This results in the TypeError.

    Simply remove super().__init__() to resolve your issue.

    This behavior was changed in #27545 (see [Lib/typing.py:1384](https://github.com/python/cpython/blob/main/Lib/typing.py#L1384)); I don't see what you are reporting as a regression, I see it as correct behavior that I would expect. Apologies if you feel differently.

    Cheers,
    Julian

    @icgood
    Copy link
    Mannequin

    icgood mannequin commented Sep 12, 2021

    Julian,

    That is certainly a workaround, however the behavior you are describing is inconsistent with PEP-544 in both word and intention. From the PEP:

    To explicitly declare that a certain class implements a given protocol, it can be used as a regular base class.

    It further describes the semantics of inheriting as "unchanged" from a "regular base class". If the semantics are "unchanged" then it should follow that super().__init__() would pass through the protocol to the object.__init__, just like a "regular base class" would if it does not override __init__.

    Furthermore, the intention of inheriting a Protocol as described in the PEP:

    Static analysis tools are expected to automatically detect that a class implements a given protocol. So while it's possible to subclass a protocol explicitly, it's not necessary to do so for the sake of type-checking.

    The purpose of adding a Protocol sub-class as an explicit base class is thus only to improve static analysis, it should *not* to modify the runtime semantics.

    Consider the case where a package maintainer wants to enhance the flexibility of their types by transitioning from using an ABC to using structural sub-typing. That simple typing change would be a breaking change to the package consumers, who must now remove a super().__init__() call.

    Ian

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    pytorchmergebot pushed a commit to pytorch/pytorch that referenced this issue Sep 29, 2022
    Remove the `_WithOp` protocol because it is not used and causes the dataclass `GraphContext` to not be able to init in some python versions.
    
    Reference to issue of dataclasses Inheriting from Protocol python/cpython#89244
    
    Pull Request resolved: #85916
    Approved by: https://github.com/BowenBao, https://github.com/abock, https://github.com/thiagocrepaldi
    mehtanirav pushed a commit to pytorch/pytorch that referenced this issue Oct 4, 2022
    Remove the `_WithOp` protocol because it is not used and causes the dataclass `GraphContext` to not be able to init in some python versions.
    
    Reference to issue of dataclasses Inheriting from Protocol python/cpython#89244
    
    Pull Request resolved: #85916
    Approved by: https://github.com/BowenBao, https://github.com/abock, https://github.com/thiagocrepaldi
    alvgaona pushed a commit to alvgaona/pytorch that referenced this issue Oct 11, 2022
    Remove the `_WithOp` protocol because it is not used and causes the dataclass `GraphContext` to not be able to init in some python versions.
    
    Reference to issue of dataclasses Inheriting from Protocol python/cpython#89244
    
    Pull Request resolved: pytorch#85916
    Approved by: https://github.com/BowenBao, https://github.com/abock, https://github.com/thiagocrepaldi
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants