This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Changed behaviour of inspect.signature() in Python 3.10
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Zac Hatfield-Dodds, gvanrossum, jack1142, kj, larry
Priority: normal Keywords:

Created on 2021-01-23 06:24 by Zac Hatfield-Dodds, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (8)
msg385524 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-01-23 06:24
Consider the following snippet, which passes on Python 3.9 and earlier:


    import inspect

    def f(x: int = None):
        pass

    print(inspect.signature(constructor))
    assert inspect.signature(constructor).parameters["a"].annotation == int


But under Python 3.10 (alpha 4), the annotation is instead read as Optional[int].  This is correct for typing.get_type_hints(), but *not* for inspect.signature().

I therefore suspect that this is an accidental side-effect from support for PEP-563 deferred evaluation of annotations.
msg385526 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-01-23 09:56
> This is correct for typing.get_type_hints(), but *not* for inspect.signature(). I therefore suspect that this is an accidental side-effect from support for PEP-563 deferred evaluation of annotations.

Spot on about PEP 563 becoming the default being the culprit!

The what's new for Python 3.10 states that inspect.signature will try to resolve types https://docs.python.org/3.10/whatsnew/3.10.html#pep-563-postponed-evaluation-of-annotations-becomes-default.

For annotations, inspect.signature internally calls get_type_hints: https://github.com/python/cpython/blob/master/Lib/inspect.py#L2194

A simple solution I can think of is to add a flag to get_type_hints to disable converting to Optional[int] for inspect.signature.

I nosied some people involved in issue38605 so that they can give their opinions on this.
msg385534 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-01-23 13:18
A closely related problem, also via https://github.com/HypothesisWorks/hypothesis/issues/2767


    import inspect

    def f(x) -> None:
        pass

    annotations = inspect.getfullargspec(f).annotations
    assert annotations == {"return": None}, annotations


Under Python 3.10, annotations is now {"return": <class 'NoneType'>}
msg385535 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-01-23 13:37
Aaaand it looks like another problem I'm having is also related:


    import inspect

    class T(typing.TypedDict):
        a: int

    print(inspect.signature(T))

was `(*args, **kwargs)` and now raises `ValueError: no signature found for builtin type <class 'dict'>`
msg385536 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-01-23 13:45
And I promise this is the last one:


    import inspect
    import typing

    def f():
        A = typing.TypeVar("A")

        def same_type_args(a: A, b: A):
            assert type(a) == type(b)

        print(inspect.signature(same_type_args))
        print(typing.get_type_hints(same_type_args))

    f()


$ python3 t.py 
(a: ~A, b: ~A)
{'a': ~A, 'b': ~A}

$ python3.10 t.py 
(a: 'A', b: 'A')
Traceback (most recent call last):
  ...
NameError: name 'A' is not defined


My apologies to everyone involved here; I'm looking forward to postponed annotation evaluation, appreciate all your work on this, and regret that I have such bizzare bugs to report <3
msg385541 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-01-23 15:40
For anyone interested, I went to do some digging on the 3 issues Zac listed:

1. Similar to the first message, this is caused by inspect.getfullarg/signature using get_type_hints in Py 3.10. get_type_hints internally converts None to NoneType.

2. I don't know what's causing that TypeDict signature to fail.

3. I'm not too sure, but maybe this is intentional according to https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-function-local-state-when-defining-annotations ? You can fix it by passing in locals() to get_type_hints (this should work all the way back to 3.6)::

import typing

def f():
    A = typing.TypeVar("A")
    def same_type_args(a: A, b: A):
        assert type(a) == type(b)
    print(typing.get_type_hints(same_type_args, localns=locals()))

>>> f()
{'a': ~A, 'b': ~A}

The whatsnew should probably be updated to mention this, this is a backwards incompatible change after all.
msg385554 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-23 17:53
Note that this may be solved in a different way *if* PEP 649 is accepted. This was discussed extensively on python-dev last week. It would roll back the stringification of PEP 563 (unless `from __future__ import annotations` is used, which would be deprecated).
msg392797 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-05-03 12:33
I think that with PEP-563 reverted, this issue has been fixed.

If we have a similar problem from e.g. PEP-649, I'll open another ticket then!
History
Date User Action Args
2022-04-11 14:59:40adminsetgithub: 87172
2021-05-03 12:33:06Zac Hatfield-Doddssetstatus: open -> closed
resolution: fixed
messages: + msg392797

stage: resolved
2021-01-29 19:53:37jack1142setnosy: + jack1142
2021-01-23 17:53:43gvanrossumsetnosy: + larry
messages: + msg385554
2021-01-23 15:40:55kjsetmessages: + msg385541
2021-01-23 13:45:35Zac Hatfield-Doddssetmessages: + msg385536
2021-01-23 13:37:58Zac Hatfield-Doddssetmessages: + msg385535
2021-01-23 13:18:37Zac Hatfield-Doddssetmessages: + msg385534
2021-01-23 09:56:07kjsetnosy: + gvanrossum, kj, BTaskaya
messages: + msg385526
2021-01-23 06:24:23Zac Hatfield-Doddscreate