Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type expression is coerced to a list of parameter arguments in substitution of ParamSpec #88964

Closed
serhiy-storchaka opened this issue Aug 1, 2021 · 5 comments
Labels
3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 44801
Nosy @gvanrossum, @ambv, @serhiy-storchaka, @miss-islington, @Fidget-Spinner
PRs
  • bpo-44801: Check arguments in substitution of ParamSpec in Callable #27585
  • [3.10] bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585) #27598
  • Dependencies
  • bpo-44793: Arguments ignored in substitution in typing.Callable
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-08-04.21:00:34.993>
    created_at = <Date 2021-08-01.07:08:58.391>
    labels = ['type-bug', 'library', '3.10', '3.11']
    title = 'Type expression is coerced to a list of parameter arguments in substitution of ParamSpec'
    updated_at = <Date 2021-08-04.21:00:34.992>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2021-08-04.21:00:34.992>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-08-04.21:00:34.993>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2021-08-01.07:08:58.391>
    creator = 'serhiy.storchaka'
    dependencies = ['44793']
    files = []
    hgrepos = []
    issue_num = 44801
    keywords = ['patch']
    message_count = 5.0
    messages = ['398687', '398688', '398842', '398925', '398953']
    nosy_count = 5.0
    nosy_names = ['gvanrossum', 'lukasz.langa', 'serhiy.storchaka', 'miss-islington', 'kj']
    pr_nums = ['27585', '27598']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue44801'
    versions = ['Python 3.10', 'Python 3.11']

    @serhiy-storchaka
    Copy link
    Member Author

    Type expression is coerced to a list of parameter arguments in substitution of ParamSpec. For example:

    >>> from typing import *
    >>> T = TypeVar('T')
    >>> P = ParamSpec('P')
    >>> C = Callable[P, T]
    >>> C[int, str]
    typing.Callable[[int], str]

    int becomes [int]. There is even a dedicated test for this.

    But it is not followed from PEP-612. Furthermore, it contradicts one of examples in the PEP:

    >>> class X(Generic[T, P]):
    ...     f: Callable[P, int]
    ...     x: T
    ... 
    >>> X[int, int]  # Should be rejected
    __main__.X[int, int]

    It makes the implementation (at least the code in bpo-44796) more complex and makes the user code more errorprone.

    @serhiy-storchaka serhiy-storchaka added stdlib Python modules in the Lib dir 3.10 only security fixes 3.11 only security fixes type-bug An unexpected behavior, bug, or error labels Aug 1, 2021
    @Fidget-Spinner
    Copy link
    Member

    Type expression is coerced to a list of parameter arguments in substitution of ParamSpec.

    It's not, only the repr is like that. Internally it's not coerced.

    >>> C[int, str]
    typing.Callable[[int], str]
    >>> C[int, str].__args__
    (<class 'int'>, <class 'str'>)

    Because Callable's correct form is Callable[[type], type] (where type is not ParamSpec or Concatenate) so the repr reflects that.

    Internally, Callable also flattens the list of args:

    >>> Callable[[int, str], int].__args__
    (<class 'int'>, <class 'str'>, <class 'int'>)

    Because __args__ *must* be hashable and immutable. Otherwise, it will not work with Union or Literal. Some time ago we tried converting to nested tuple. But that was too difficult (and backwards incompatible) because every other typing operation expected __args__ to contain types, not a tuple.

    @serhiy-storchaka
    Copy link
    Member Author

    Thank you Ken Jin. So the problem is the __args__ (x, str) is interpreted differently depending on x, and after substituting x the interpretation can be changed. If x was ParamSpec, it was interpreted in one way, but if it becomes int, it is now interpreted in other way.

    The solution is to forbid substitution of P with wrong values (not parameters expression). Some normalization is also needed, before and after substitution.

    Other related example is:

    >>> from typing import *
    >>> P = ParamSpec("P")
    >>> class Z(Generic[P]): pass
    ... 
    >>> A = Z[[int]]
    >>> B = Z[int]
    >>> A
    __main__.Z[(<class 'int'>,)]
    >>> B
    __main__.Z[int]
    >>> A.__args__
    ((<class 'int'>,),)
    >>> B.__args__
    (<class 'int'>,)

    It is expected that A and B should the same.

    @ambv
    Copy link
    Contributor

    ambv commented Aug 4, 2021

    New changeset 3875a69 by Serhiy Storchaka in branch 'main':
    bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
    3875a69

    @miss-islington
    Copy link
    Contributor

    New changeset 536e35a by Miss Islington (bot) in branch '3.10':
    bpo-44801: Check arguments in substitution of ParamSpec in Callable (GH-27585)
    536e35a

    @ambv ambv closed this as completed Aug 4, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants