classification
Title: Type expression is coerced to a list of parameter arguments in substitution of ParamSpec
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: 44793 Superseder:
Assigned To: Nosy List: gvanrossum, kj, lukasz.langa, miss-islington, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2021-08-01 07:08 by serhiy.storchaka, last changed 2021-08-04 21:00 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27585 merged serhiy.storchaka, 2021-08-03 18:12
PR 27598 merged miss-islington, 2021-08-04 18:07
Messages (5)
msg398687 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-01 07:08
Type expression is coerced to a list of parameter arguments in substitution of ParamSpec. For example:

>>> from typing import *
>>> T = TypeVar('T')
>>> P = ParamSpec('P')
>>> C = Callable[P, T]
>>> C[int, str]
typing.Callable[[int], str]

int becomes [int]. There is even a dedicated test for this.

But it is not followed from PEP 612. Furthermore, it contradicts one of examples in the PEP:

>>> class X(Generic[T, P]):
...     f: Callable[P, int]
...     x: T
... 
>>> X[int, int]  # Should be rejected
__main__.X[int, int]

It makes the implementation (at least the code in issue44796) more complex and makes the user code more errorprone.
msg398688 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-08-01 07:55
> Type expression is coerced to a list of parameter arguments in substitution of ParamSpec.

It's not, only the repr is like that. Internally it's not coerced.

>>> C[int, str]
typing.Callable[[int], str]
>>> C[int, str].__args__
(<class 'int'>, <class 'str'>)

Because Callable's correct form is Callable[[type], type] (where type is not ParamSpec or Concatenate) so the repr reflects that.

Internally, Callable also flattens the list of args:

>>> Callable[[int, str], int].__args__
(<class 'int'>, <class 'str'>, <class 'int'>)

Because __args__ *must* be hashable and immutable. Otherwise, it will not work with Union or Literal. Some time ago we tried converting to nested tuple. But that was too difficult (and backwards incompatible) because every other typing operation expected __args__ to contain types, not a tuple.
msg398842 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-03 17:46
Thank you Ken Jin. So the problem is the __args__ (x, str) is interpreted differently depending on x, and after substituting x the interpretation can be changed. If x was ParamSpec, it was interpreted in one way, but if it becomes int, it is now interpreted in other way.

The solution is to forbid substitution of P with wrong values (not parameters expression). Some normalization is also needed, before and after substitution.

Other related example is:

>>> from typing import *
>>> P = ParamSpec("P")
>>> class Z(Generic[P]): pass
... 
>>> A = Z[[int]]
>>> B = Z[int]
>>> A
__main__.Z[(<class 'int'>,)]
>>> B
__main__.Z[int]
>>> A.__args__
((<class 'int'>,),)
>>> B.__args__
(<class 'int'>,)

It is expected that A and B should the same.
msg398925 - (view) Author: Ɓukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-04 18:07
New changeset 3875a6954741065b136650db67ac533bc70a3eac by Serhiy Storchaka in branch 'main':
bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
https://github.com/python/cpython/commit/3875a6954741065b136650db67ac533bc70a3eac
msg398953 - (view) Author: miss-islington (miss-islington) Date: 2021-08-04 20:36
New changeset 536e35ae6a2555a01f4b51a68ad71dbf7923536d by Miss Islington (bot) in branch '3.10':
bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
https://github.com/python/cpython/commit/536e35ae6a2555a01f4b51a68ad71dbf7923536d
History
Date User Action Args
2021-08-04 21:00:34lukasz.langasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-08-04 20:36:34miss-islingtonsetmessages: + msg398953
2021-08-04 18:07:09miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26099
2021-08-04 18:07:08lukasz.langasetnosy: + lukasz.langa
messages: + msg398925
2021-08-03 18:12:53serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request26088
2021-08-03 17:46:04serhiy.storchakasetmessages: + msg398842
2021-08-01 07:55:02kjsetmessages: + msg398688
2021-08-01 07:09:52serhiy.storchakasetdependencies: + Arguments ignored in substitution in typing.Callable
type: behavior
components: + Library (Lib)
versions: + Python 3.10, Python 3.11
2021-08-01 07:08:58serhiy.storchakacreate