Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameter substitution in the union type does not work with typing.Union #88819

Closed
serhiy-storchaka opened this issue Jul 16, 2021 · 8 comments
Closed
Labels
3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 44653
Nosy @gvanrossum, @ambv, @serhiy-storchaka, @miss-islington, @uriyyo, @Fidget-Spinner
PRs
  • bpo-44653: Support typing.Union in parameter substitution of the union type #27232
  • bpo-44653: Support typing types in parameter substitution in the union type. #27247
  • [3.10] bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247) #27296
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-07-22.22:21:37.910>
    created_at = <Date 2021-07-16.13:32:27.733>
    labels = ['interpreter-core', 'type-bug', '3.10', '3.11']
    title = 'Parameter substitution in the union type does not work with typing.Union'
    updated_at = <Date 2021-07-22.22:34:04.923>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2021-07-22.22:34:04.923>
    actor = 'gvanrossum'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-07-22.22:21:37.910>
    closer = 'lukasz.langa'
    components = ['Interpreter Core']
    creation = <Date 2021-07-16.13:32:27.733>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44653
    keywords = ['patch']
    message_count = 8.0
    messages = ['397624', '397803', '397827', '397828', '398009', '398011', '398012', '398014']
    nosy_count = 6.0
    nosy_names = ['gvanrossum', 'lukasz.langa', 'serhiy.storchaka', 'miss-islington', 'uriyyo', 'kj']
    pr_nums = ['27232', '27247', '27296']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44653'
    versions = ['Python 3.10', 'Python 3.11']

    @serhiy-storchaka
    Copy link
    Member Author

    >>> import typing
    >>> T = typing.TypeVar('T')
    >>> (int | T)[typing.Union[str, list]]
    NotImplemented

    See also bpo-44633. But in this case the expected result is int | str | list or typing.Union[init, str, list].

    @serhiy-storchaka serhiy-storchaka added 3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Jul 16, 2021
    @Fidget-Spinner
    Copy link
    Member

    @serhiy, this doesn't just affect typing.Union, it seems that the rest of the typing types don't substitute:

    >>> (int | T)[typing.List[str]]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union arg must be a type, got typing.List[str]

    We should probably loosen the check during the union_getitem substitution -- no need to raise the TypeError or check for is_union, just blindly replace the TypeVar. We already do this for types.GenericAlias:

    >>> list[T][1]
    list[1]

    Or if you want to continue checking, maybe checking for PyCallable_Check(obj) in substitution is enough - typing internally accepts callable(o) too:

    https://github.com/python/cpython/blob/3.10/Lib/typing.py#L146

    @serhiy-storchaka
    Copy link
    Member Author

    There is also problem with other typing types:

    >>> (int | T)[typing.List]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union argument must be a type, got typing.List
    >>> (int | T)[typing.List[int]]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union argument must be a type, got typing.List[int]
    >>> (int | T)[typing.Hashable]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union argument must be a type, got typing.Hashable
    >>> (int | T)[typing.Callable[[int], str]]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union argument must be a type, got typing.Callable[[int], str]
    >>> (int | T)[typing.ParamSpec('P')]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: Each union argument must be a type, got ~P

    Despite the fact that they support the | operator.

    We can add one by one special support of different types supporting the | operator (ParamSpec, _GenericAlias, _CallableGenericAlias, _UnionGenericAlias, _LiteralGenericAlias, _ConcatenateGenericAlias, _AnnotatedAlias, _SpecialGenericAlias, _CallableType, _TupleType), but it is cumbersome and errorprone. We will need to synchronize code of unionobject.c with typing every time we add new kind of types.

    PR 27247 uses more general approach. It calls the | operator for arguments after substitution. So all types which support the | operator are now automatically supported. But the result of parameter substitution can now be typing.Union instead of types.Union.

    >>> import typing
    >>> import collections.abc
    >>> T = typing.TypeVar('T')
    >>> (int | T)[list]
    int | list
    >>> (int | T)[typing.List]
    typing.Union[int, typing.List]
    >>> (int | T)[list[int]]
    int | list[int]
    >>> (int | T)[typing.List[int]]
    typing.Union[int, typing.List[int]]
    >>> (int | T)[collections.abc.Hashable]
    int | collections.abc.Hashable
    >>> (int | T)[typing.Hashable]
    typing.Union[int, typing.Hashable]
    >>> (int | T)[collections.abc.Sequence[int]]
    int | collections.abc.Sequence[int]
    >>> (int | T)[typing.Sequence[int]]
    typing.Union[int, typing.Sequence[int]]
    >>> (int | T)[collections.abc.Callable[[int], str]]
    int | collections.abc.Callable[[int], str]
    >>> (int | T)[typing.Callable[[int], str]]
    typing.Union[int, typing.Callable[[int], str]]
    >>> (int | T)[typing.TypeVar('S')]
    int | ~S
    >>> (int | T)[typing.ParamSpec('P')]
    typing.Union[int, ~P]
    >>> (int | T)[str | list]
    int | str | list
    >>> (int | T)[typing.Union[str, list]]
    typing.Union[int, str, list]

    @gvanrossum
    Copy link
    Member

    I need someone else to own this, sorry.

    @ambv
    Copy link
    Contributor

    ambv commented Jul 22, 2021

    New changeset 2e3744d by Serhiy Storchaka in branch 'main':
    bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247)
    2e3744d

    @ambv
    Copy link
    Contributor

    ambv commented Jul 22, 2021

    New changeset 21db59f by Miss Islington (bot) in branch '3.10':
    bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247) (bpo-27296)
    21db59f

    @ambv
    Copy link
    Contributor

    ambv commented Jul 22, 2021

    Thanks Serhiy for the report and the fix, and Ken for reviews.

    @ambv ambv closed this as completed Jul 22, 2021
    @ambv ambv closed this as completed Jul 22, 2021
    @gvanrossum
    Copy link
    Member

    Thanks for taking over here, Ł!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants