This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add __parameters__ and __getitem__ in TypeVar and ParamSpec
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: 44794 Superseder:
Assigned To: Nosy List: AlexWaygood, JelleZijlstra, gvanrossum, kj, serhiy.storchaka, sobolevn
Priority: normal Keywords: patch

Created on 2021-07-31 13:31 by serhiy.storchaka, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 27511 open serhiy.storchaka, 2021-07-31 13:34
PR 31143 merged serhiy.storchaka, 2022-02-05 13:58
Messages (9)
msg398640 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-31 13:31
Adding __parameters__ and __getitem__ in TypeVar and ParamSpec allows to generalize and simplify the code (especially the C code) and allows to add more runtime checks. It may open ways for further simplification.

Unfortunately it is not compatible with issue44098, so the latter changes should be reverted.
msg411555 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-01-25 05:40
The linked PR makes `T = TypeVar("T"); T[int]` work. That's not currently meaningful syntax to the type checker, but we may want it in the future for higher-kinded types. I would rather not make this change without a clear static typing use case in mind.
msg412464 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-02-03 20:23
It does not have a use case of T[int] in mind. It is an implementation detail which simplifies the code, makes runtime checks more strict and makes types.GenericAlias more similar to typing._GenericAlias.

For example, currently:

>>> A = List[T]
>>> B = list[T]
>>> A[None]
typing.List[NoneType]
>>> B[None]
list[None]
>>> A['X']
typing.List[ForwardRef('X')]
>>> B['X']
list['X']

With the proposed change:

>>> B[None]
list[NoneType]
>>> B['X']
list[ForwardRef('X')]

Meanless expressions like A[42], A[Final], A[Final[int]], A[ClassVar], A[ClassVar[int]], etc which are silently accepted will now be errors.

The code simplification (especially for the C code) is my primary goal, and the rest is a nice side effect. It is possible to add more strict runtime checks and conversions without making TypeVar and ParamSpec subscriptable, but the code will not be so simple.
msg412483 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-02-03 22:52
I don't think that changing list[None] to list[NoneType] in the output is an improvement. I do appreciate the reduction in C code though!
msg412484 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-02-03 23:10
I'd also be wary about making changes that introduce additional runtime checks. As we've seen with the PEP 612 and 646 implementations, those can impede future iteration on typing.
msg412575 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-02-05 14:21
I have created an alternative PR 31143 which adds special method __typing_subst__ in TypeVar and ParamSpec instead of __parameters__ and __getitem__. It has almost the same effect except making TypeVar and ParamSpec and subscriptable. The Python code in typing.py and _collections_abc.py is simplified, but the C code is little changed.

But on other hand, the reasons for PR 27511:

1. We already implemented __getitem__ for a Concatenate alias, although it is never documented. It is not even necessary, it is a pure implementation detail which makes the code for nested variables substitution simpler. Adding __parameters__ and __getitem__ in TypeVar and ParamSpec would have the same effect on simplification of the code.

2. There is a use case for subscription of TypeVar. For example:

if version < 2:
   RetType = T
elif version < 3:
   RetType = Optional[T]
else:
   RetType = Optional[T] | Literal[Unknown]

def get_int(precision: int) -> RetType[int]: ...
def get_str(encoding: str) -> RetType[str]: ...
def get_any(converter: Callable[[str], T]) -> RetType[T]: ...
msg412704 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-02-07 02:13
Super subtle stuff. Tonight I do not have time to really dive into this, but I think Serhiy is on to something. KJ or Jelle, do you have the guts to dive in here?
msg414895 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-03-11 08:47
New changeset b6a5d8590c4bfe4553d796b36af03bda8c0d5af5 by Serhiy Storchaka in branch 'main':
bpo-44796: Unify TypeVar and ParamSpec substitution (GH-31143)
https://github.com/python/cpython/commit/b6a5d8590c4bfe4553d796b36af03bda8c0d5af5
msg414992 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2022-03-12 13:54
PR 31143 simplified some code without changing the common API:

 Objects/genericaliasobject.c | 46 ++++++++++++++--------------------------------
 1 file changed, 14 insertions(+), 32 deletions(-)
 Lib/_collections_abc.py | 63 +++++++++------------------------------------------------------
 1 file changed, 9 insertions(+), 54 deletions(-)

PR 27511 will simplify it even more:

 Objects/genericaliasobject.c | 60 ++++++++++++++++++------------------------------------------
 1 file changed, 18 insertions(+), 42 deletions(-)

(Lib/typing.py will be simplified too, even if this does not reduce the number of lines).
History
Date User Action Args
2022-04-11 14:59:48adminsetgithub: 88959
2022-03-12 13:54:12serhiy.storchakasetmessages: + msg414992
2022-03-11 08:47:56serhiy.storchakasetmessages: + msg414895
2022-02-07 02:13:00gvanrossumsetmessages: + msg412704
2022-02-05 14:21:07serhiy.storchakasetmessages: + msg412575
2022-02-05 13:58:29serhiy.storchakasetpull_requests: + pull_request29321
2022-02-03 23:10:49JelleZijlstrasetmessages: + msg412484
2022-02-03 23:02:06AlexWaygoodsetnosy: + sobolevn, AlexWaygood
2022-02-03 22:52:07gvanrossumsetmessages: + msg412483
2022-02-03 20:23:24serhiy.storchakasetmessages: + msg412464
2022-01-25 05:40:33JelleZijlstrasetnosy: + JelleZijlstra
messages: + msg411555
2021-07-31 17:50:42serhiy.storchakasetmessages: - msg398655
2021-07-31 17:50:22serhiy.storchakasetmessages: + msg398655
2021-07-31 13:34:46serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request26025
2021-07-31 13:32:02serhiy.storchakasetdependencies: + Merge tests for typing.Callable and collection.abc.Callable
2021-07-31 13:31:29serhiy.storchakacreate