Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot extend collections ABCs with protocol #89187

Closed
anuppari mannequin opened this issue Aug 27, 2021 · 8 comments
Closed

Cannot extend collections ABCs with protocol #89187

anuppari mannequin opened this issue Aug 27, 2021 · 8 comments
Assignees
Labels
3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error

Comments

@anuppari
Copy link
Mannequin

anuppari mannequin commented Aug 27, 2021

BPO 45024
Nosy @gvanrossum, @rhettinger, @JelleZijlstra, @pablogsal, @miss-islington, @Fidget-Spinner, @anuppari
PRs
  • bpo-45024 and bpo-23864: Document how interface testing works with the collections ABCs #28218
  • [3.10] bpo-45024 and bpo-23864: Document how interface testing works with the collections ABCs (GH-28218) #28266
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/rhettinger'
    closed_at = <Date 2021-09-10.04:00:18.168>
    created_at = <Date 2021-08-27.01:29:49.472>
    labels = ['3.11', 'type-bug', '3.10', 'docs']
    title = 'Cannot extend collections ABCs with protocol'
    updated_at = <Date 2021-10-04.19:18:42.877>
    user = 'https://github.com/anuppari'

    bugs.python.org fields:

    activity = <Date 2021-10-04.19:18:42.877>
    actor = 'pablogsal'
    assignee = 'rhettinger'
    closed = True
    closed_date = <Date 2021-09-10.04:00:18.168>
    closer = 'rhettinger'
    components = ['Documentation']
    creation = <Date 2021-08-27.01:29:49.472>
    creator = 'anuppari'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45024
    keywords = ['patch']
    message_count = 8.0
    messages = ['400387', '400465', '400502', '400924', '400953', '401547', '401550', '403162']
    nosy_count = 8.0
    nosy_names = ['gvanrossum', 'rhettinger', 'docs@python', 'JelleZijlstra', 'pablogsal', 'miss-islington', 'kj', 'anuppari']
    pr_nums = ['28218', '28266']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue45024'
    versions = ['Python 3.10', 'Python 3.11']

    @anuppari
    Copy link
    Mannequin Author

    anuppari mannequin commented Aug 27, 2021

    Since the container ABCs are normal classes, and Protocol cannot subclass normal classes, there's no way to create a protocol that extends the ABCs without explicitly listing out all the methods needed for the collection. e.g., can't do this:

    from typing import Iterable, Protocol
    
    class IterableWithMethod(Iterable, Protocol):
        def method(self) -> None: pass

    Since the ABCs don't provide any default implementations (I think?), maybe they should just be defined as runtime checkable protocols instead of ABCs?

    @anuppari anuppari mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Aug 27, 2021
    @Fidget-Spinner
    Copy link
    Member

    Thanks for the bug report Anup. I'm mildly confused by what you meant, so please help me here:

    I interpreted your request as wanting any type that implements method and __iter__ to be considered a subtype of IterableWithMethod. But this already works at runtime and with mypy.

    Are you perhaps saying that you want issubclass(Foo, IterableWithMethod) to work?

    @Fidget-Spinner Fidget-Spinner removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Aug 28, 2021
    @anuppari
    Copy link
    Mannequin Author

    anuppari mannequin commented Aug 28, 2021

    Yeah, you got the gist of what I meant, though, I guess that was a confusing example since there are apparently protocols for Iterable. But not for many of the other collection ABCs such as Sequence, e.g., the following doesn't work:

    from typing import Sequence, Protocol
    
    class SequenceWithMethod(Sequence, Protocol):
        def method(self) -> None: pass

    In which case, anything that implements method, __getitem__, __len__, __contains__, __iter__, __reversed__, index, and count would be considered a subtype of SequenceWithMethod

    @Fidget-Spinner
    Copy link
    Member

    I suspect this may be intentional. See related issues https://bugs.python.org/issue35190 and https://bugs.python.org/issue23864.

    Copying verbatim from what Ivan said in the first issue:

    "
    The separation may look arbitrary, but the idea is quite simple. Only those classes with few methods support structural checks. Those classes have few independent abstract methods (or even just one method), while in classes with large APIs like Sequence, the methods are not logically independent, so you can't say a class is 100% a Sequence even if types/signatures of all methods are correct, because e.g. __contains__() and index() should behave in agreement with __getitem__().
    "

    @rhettinger
    Copy link
    Contributor

    The docs need to clarify which ABCs can satisfy isinstance() based solely on hasattr() logic and which ABCs require registration or direct subclassing.

    The docs should also explain why the more complex ABCs can't reliably be recognized based on the presence of desired method names.

    One of the original motivating reasons for the ABCs was that the presence of __getitem__ was insufficient for determining whether a class was intended to be a Mapping or a Sequence. Accordingly, those classes have to subclass from an ABC or register with the ABC in order to declare intent.

    @rhettinger rhettinger added docs Documentation in the Doc dir and removed stdlib Python modules in the Lib dir labels Sep 2, 2021
    @rhettinger rhettinger added docs Documentation in the Doc dir and removed stdlib Python modules in the Lib dir labels Sep 2, 2021
    @rhettinger
    Copy link
    Contributor

    New changeset 62fa613 by Raymond Hettinger in branch 'main':
    bpo-45024 and bpo-23864: Document how interface testing works with the collections ABCs (GH-28218)
    62fa613

    @rhettinger
    Copy link
    Contributor

    New changeset 89edd18 by Miss Islington (bot) in branch '3.10':
    bpo-45024 and bpo-23864: Document how interface testing works with the collections ABCs (GH-28218) (GH-28266)
    89edd18

    @rhettinger rhettinger added 3.10 only security fixes 3.11 only security fixes labels Sep 10, 2021
    @rhettinger rhettinger assigned rhettinger and unassigned docspython Sep 10, 2021
    @rhettinger rhettinger added 3.10 only security fixes 3.11 only security fixes labels Sep 10, 2021
    @rhettinger rhettinger self-assigned this Sep 10, 2021
    @pablogsal
    Copy link
    Member

    New changeset 9d8a64a by Pablo Galindo (Miss Islington (bot)) in branch '3.10':
    bpo-45024 and bpo-23864: Document how interface testing works with the collections ABCs (GH-28218) (GH-28266)
    9d8a64a

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants