classification
Title: Parameter substitution in the union type does not work with typing.Union
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, lukasz.langa, miss-islington, serhiy.storchaka, uriyyo
Priority: normal Keywords: patch

Created on 2021-07-16 13:32 by serhiy.storchaka, last changed 2021-07-22 22:34 by gvanrossum. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27232 closed serhiy.storchaka, 2021-07-18 16:35
PR 27247 merged serhiy.storchaka, 2021-07-19 17:50
PR 27296 merged miss-islington, 2021-07-22 21:57
Messages (8)
msg397624 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-16 13:32
>>> import typing
>>> T = typing.TypeVar('T')
>>> (int | T)[typing.Union[str, list]]
NotImplemented

See also issue44633. But in this case the expected result is int | str | list or typing.Union[init, str, list].
msg397803 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-07-19 14:56
@Serhiy, this doesn't just affect typing.Union, it seems that the rest of the typing types don't substitute:

>>> (int | T)[typing.List[str]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union arg must be a type, got typing.List[str]

We should probably loosen the check during the union_getitem substitution -- no need to raise the TypeError or check for is_union, just blindly replace the TypeVar. We already do this for types.GenericAlias:

>>> list[T][1]
list[1]

Or if you want to continue checking, maybe checking for PyCallable_Check(obj) in substitution is enough - typing internally accepts callable(o) too:

https://github.com/python/cpython/blob/3.10/Lib/typing.py#L146
msg397827 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-19 18:15
There is also problem with other typing types:

>>> (int | T)[typing.List]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union argument must be a type, got typing.List
>>> (int | T)[typing.List[int]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union argument must be a type, got typing.List[int]
>>> (int | T)[typing.Hashable]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union argument must be a type, got typing.Hashable
>>> (int | T)[typing.Callable[[int], str]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union argument must be a type, got typing.Callable[[int], str]
>>> (int | T)[typing.ParamSpec('P')]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Each union argument must be a type, got ~P

Despite the fact that they support the | operator.

We can add one by one special support of different types supporting the | operator (ParamSpec, _GenericAlias, _CallableGenericAlias, _UnionGenericAlias, _LiteralGenericAlias, _ConcatenateGenericAlias, _AnnotatedAlias, _SpecialGenericAlias, _CallableType, _TupleType), but it is cumbersome and errorprone. We will need to synchronize code of unionobject.c with typing every time we add new kind of types.

PR 27247 uses more general approach. It calls the | operator for arguments after substitution. So all types which support the | operator are now automatically supported. But the result of parameter substitution can now be typing.Union instead of types.Union.

>>> import typing
>>> import collections.abc
>>> T = typing.TypeVar('T')
>>> (int | T)[list]
int | list
>>> (int | T)[typing.List]
typing.Union[int, typing.List]
>>> (int | T)[list[int]]
int | list[int]
>>> (int | T)[typing.List[int]]
typing.Union[int, typing.List[int]]
>>> (int | T)[collections.abc.Hashable]
int | collections.abc.Hashable
>>> (int | T)[typing.Hashable]
typing.Union[int, typing.Hashable]
>>> (int | T)[collections.abc.Sequence[int]]
int | collections.abc.Sequence[int]
>>> (int | T)[typing.Sequence[int]]
typing.Union[int, typing.Sequence[int]]
>>> (int | T)[collections.abc.Callable[[int], str]]
int | collections.abc.Callable[[int], str]
>>> (int | T)[typing.Callable[[int], str]]
typing.Union[int, typing.Callable[[int], str]]
>>> (int | T)[typing.TypeVar('S')]
int | ~S
>>> (int | T)[typing.ParamSpec('P')]
typing.Union[int, ~P]
>>> (int | T)[str | list]
int | str | list
>>> (int | T)[typing.Union[str, list]]
typing.Union[int, str, list]
msg397828 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-19 18:20
I need someone else to own this, sorry.
msg398009 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-22 21:57
New changeset 2e3744d50b6e30ea24351e55b4352dcc58fd469e by Serhiy Storchaka in branch 'main':
bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247)
https://github.com/python/cpython/commit/2e3744d50b6e30ea24351e55b4352dcc58fd469e
msg398011 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-22 22:18
New changeset 21db59fc75b6ebb01bf120a8e5930fe032174f73 by Miss Islington (bot) in branch '3.10':
bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247) (#27296)
https://github.com/python/cpython/commit/21db59fc75b6ebb01bf120a8e5930fe032174f73
msg398012 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-22 22:21
Thanks Serhiy for the report and the fix, and Ken for reviews.
msg398014 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-22 22:34
Thanks for taking over here, Ł!
History
Date User Action Args
2021-07-22 22:34:04gvanrossumsetmessages: + msg398014
2021-07-22 22:21:37lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg398012

stage: patch review -> resolved
2021-07-22 22:18:57lukasz.langasetmessages: + msg398011
2021-07-22 21:57:17miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25839
2021-07-22 21:57:10lukasz.langasetnosy: + lukasz.langa
messages: + msg398009
2021-07-19 18:20:39gvanrossumsetmessages: + msg397828
2021-07-19 18:15:58serhiy.storchakasetmessages: + msg397827
2021-07-19 17:50:08serhiy.storchakasetpull_requests: + pull_request25795
2021-07-19 14:56:50kjsetmessages: + msg397803
2021-07-18 16:35:30serhiy.storchakasetpull_requests: + pull_request25780
2021-07-18 16:31:20serhiy.storchakasetpull_requests: - pull_request25757
2021-07-17 15:16:33uriyyosetpull_requests: + pull_request25757
2021-07-17 07:02:52serhiy.storchakasetpull_requests: - pull_request25745
2021-07-17 07:01:36uriyyosetkeywords: + patch
nosy: + uriyyo

pull_requests: + pull_request25745
stage: patch review
2021-07-16 13:32:27serhiy.storchakacreate