This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Top / function argument level ClassVar should not be allowed during `get_type_hints()`
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, lukasz.langa, sobolevn
Priority: normal Keywords: patch

Created on 2021-09-24 13:44 by sobolevn, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 28563 closed sobolevn, 2021-09-25 10:35
Messages (7)
msg402563 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-09-24 13:44
This code currently does not raise any issues:

```python
# contents of `ex.py`
from typing import ClassVar

a: ClassVar[int] = 1
def b(c: ClassVar[int]): ...
```

And then: 
1. Module: `python -c 'import ex; from typing import get_type_hints; print(get_type_hints(ex))'`
2. Function argument: `python -c 'import ex; from typing import get_type_hints; print(get_type_hints(ex.b))'`

It should not be allowed. Currently, the same with `from __future__ import annotations` does correctly raise `TypeError: typing.ClassVar[int] is not valid as type argument`

Related: https://github.com/python/cpython/pull/28279

I will send a PR with the fix shortly.
msg402628 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-09-25 14:51
Hm. IMO it is up to static type checkers to reject bad use of types. The runtime should be relaxed so type checkers can develop new features assigning new semantics to such constructs.
msg402631 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-09-25 15:37
This makes ``get_type_hints`` quite opinionated and backwards incompatible. The first line the docs says "This is often the same as obj.__annotations__".

I also agree with Guido. FYI, we've tried to *reduce* usage of `_type_check` in new features since version 3.10. We've found that they make `typing_extensions` operability with `typing` worse. For example, the whole of PEP 612 in `typing_extensions` required lots of hacks to work with typing's Callable.

If you're really vouching for this change. We probably need a 2 major version deprecation period (with warnings telling the user their current type hints are invalid) *plus* asking typing-sig/python-dev if the runtime type hints users will be affected. See [PEP 387](https://www.python.org/dev/peps/pep-0387/) for more info.

On the other hand, we now have a less-opinionated alternative to ``get_type_hints`` with ``inspect.get_annotations``, but it can't do as much (like resolving ForwardRef).
msg402636 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-09-25 17:30
Just to bring in some context for this discussion: https://github.com/python/cpython/pull/28279

We have dicussed in the PR above that current implementation depends on existing of `__future__` import of `annotations`.

With `from __future__ import annotations`:
1. Top level class var will be raise `TypeError`
2. Final and ClassVar function arguments will raise `TypeError`
3. Annotations like `x: 1` will raise `TypeError: Forward references must evaluate to types. Got 1.`

Without `from __future import annotations`it accepts all the cases above.

I share two opposite views on this change:
1. By adding the same `_type_check` logic we can reduce the inconsitency between two versions. It would be easier for users to migrate from one version to another.
2. By adding this feature, we would more aggresively force some specific semantics for builtin types. Which can possibly be problematic, for example I've tried to add `Final` annotations to function arguments in my own type checker. It is something current semantics does not support. Moreover, it can cause incompatibility to existing users.

My opinion is that consitency is more important here. Because I received quite a lot of bug reports from users involving different behavior with / without `__future__` import of `annotations`.

But, I understand that it is not up to me to decide :)
msg402640 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-09-25 18:16
> 1. By adding the same `_type_check` logic we can reduce the inconsistency between two versions. It would be easier for users to migrate from one version to another.

That's definitely a plus. But please consider this hypothetical scenario:

- Let's say some typing code has _type_check in 3.8
- In 3.12, we decide to give type hinting with ints (like `x: 1`) some meaning (this has been proposed before in typing-sig).
- The new construct arrives in 3.12. We backport to typing_extensions, which *must* support 3.8's typing module.
- Now the new addition to 3.12 typing_extensions doesn't work with 3.8 typing because of _type_check, and we need to hack around it.
- The hacks may eventually lead to real behavioral inconsistencies between typing_extensions and typing.

If this sounds familiar, that's because something similar has already happened! Callable and Generic had too strict checking in typing (requiring Callable's args to be a tuple of types always) before 3.10. So typing_extensions doesn't support the full PEP 612 syntax :( https://github.com/python/typing/tree/master/typing_extensions#other-notes-and-limitations.

Stricter runtime type checking makes growing the typing syntax harder. For the newer types, we try to defer checking to 3rd-party libraries.

> Because I received quite a lot of bug reports from users involving different behavior with / without `__future__` import of `annotations`.

Your perspective as a runtime typing library dev is very valuable. Thank you for your feedback! IMO, `from __future__ import annotations` becoming the default in 3.11 is still undecided. It was yanked from 3.10 for breaking pydantic and FastAPI, and I doubt it will land in 3.11 for the same reasons. It seems the current consensus is to use that import for code with purely static typing, and not use it with runtime typing libraries.
msg402641 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-09-25 18:39
Thank you, Ken Jin! Very clear explanation!

> It seems the current consensus is to use that import for code with purely static typing, and not use it with runtime typing libraries.

Is it an official position? Or just a common usage pattern?
What can we do to make this more widely known?

Maybe we can formulate something like "When to use: raw, string, future annotations" in the typing docs (https://docs.python.org/3/library/typing.html)?
msg402671 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-09-26 17:09
> Is it an official position?

No, we need to decide what will happen to PEP 563 and PEP 649 in Python 3.11.
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89446
2021-09-26 17:09:09lukasz.langasetnosy: + lukasz.langa
messages: + msg402671
2021-09-25 18:39:40sobolevnsetmessages: + msg402641
2021-09-25 18:16:03kjsetmessages: + msg402640
2021-09-25 17:30:55sobolevnsetmessages: + msg402636
2021-09-25 15:37:16kjsetnosy: + kj
messages: + msg402631
2021-09-25 14:51:15gvanrossumsetnosy: + gvanrossum
messages: + msg402628
2021-09-25 10:35:53sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26946
2021-09-24 13:44:07sobolevncreate