classification
Title: TypedDict inheritance doesn't work with get_type_hints and postponed evaluation of annotations across modules
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, keithblaha, levkivskyi
Priority: normal Keywords:

Created on 2020-07-08 23:47 by keithblaha, last changed 2020-07-16 17:56 by keithblaha.

Messages (3)
msg373360 - (view) Author: Keith Blaha (keithblaha) Date: 2020-07-08 23:47
Copied from https://github.com/python/typing/issues/737

I came across this issue while using inheritance to express required keys in a TypedDict, as is recommended by the docs.

It's most easily explained by a minimal example I cooked up. Let's say we have a module foo.py:

from __future__ import annotations

from typing import Optional
from typing_extensions import TypedDict

class Foo(TypedDict):
    a: Optional[int]

And another module bar.py:

from __future__ import annotations

from typing import get_type_hints
from foo import Foo

class Bar(Foo, total=False):
    b: int

print(get_type_hints(Bar))

Note that both foo.py and bar.py have adopted postponed evaluation of annotations (PEP 563) by using the __future__ import.

If we execute bar.py, we get the error message NameError: name 'Optional' is not defined.

This is due to the combination of:

    get_type_hints relies on the MRO to resolve types: https://github.com/python/cpython/blob/3.7/Lib/typing.py#L970
    TypedDict does not preserve the original bases, so Foo is not in the MRO for Bar:

    typing/typing_extensions/src_py3/typing_extensions.py

    Line 1652 in d79edde

     tp_dict = super(_TypedDictMeta, cls).__new__(cls, name, (dict,), ns) 

Thus, get_type_hints is unable to resolve the types for annotations that are only imported in foo.py.

I ran this example using typing_extensions 3.7.4.2 (released via #709) and Python 3.7.3, but it seems like this would be an issue using the current main branches of both repositories as well.

I'm wondering what the right approach is to tackling this issue. It is of course solvable by defining Bar in foo.py instead, but it isn't ideal or intuitive to always need to inherit from a TypedDict in the same module.

I was thinking that similarly to __required_keys__ and __optional_keys__, the TypedDict could preserve its original bases in a new dunder attribute, and get_type_hints could work off of that instead of MRO when it is dealing with a TypedDict. I would be willing to contribute the PRs to implement this if the design is acceptable, but am open to other ideas as well.
msg373752 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2020-07-16 10:12
> I was thinking that similarly to __required_keys__ and __optional_keys__, the TypedDict could preserve its original bases in a new dunder attribute, and get_type_hints could work off of that instead of MRO when it is dealing with a TypedDict. I would be willing to contribute the PRs to implement this if the design is acceptable

TBH this is not very elegant, but I think you can go ahead with this (at least as a quick fix) since I don't see a better solution yet.
msg373758 - (view) Author: Keith Blaha (keithblaha) Date: 2020-07-16 17:56
> TBH this is not very elegant, but I think you can go ahead with this (at least as a quick fix) since I don't see a better solution yet.

Agreed, given that the current workaround of implementing them in the same module works I think I will stick with that while this is brainstormed further.
History
Date User Action Args
2020-07-16 17:56:36keithblahasetmessages: + msg373758
2020-07-16 10:12:25levkivskyisetmessages: + msg373752
2020-07-08 23:51:53gvanrossumsetnosy: + gvanrossum, levkivskyi
2020-07-08 23:47:38keithblahacreate