This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: typing.Literal[True] is implicitly converted to typing.Literal[1]
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, miss-islington, rhettinger, serhiy.storchaka, sobolevn, vanburgerberg
Priority: normal Keywords: patch

Created on 2021-10-30 15:33 by vanburgerberg, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29333 closed sobolevn, 2021-10-30 17:29
PR 29334 merged serhiy.storchaka, 2021-10-30 18:17
PR 29339 merged sobolevn, 2021-10-31 08:00
PR 29340 merged miss-islington, 2021-10-31 08:22
PR 29342 merged serhiy.storchaka, 2021-10-31 08:41
Messages (9)
msg405371 - (view) Author: Van Burgerberg (vanburgerberg) Date: 2021-10-30 15:33
When you create `Literal[1]` annotation and then create `Literal[True]` annotation, in the second case you will actually get `Literal[1]` instead. This is happening because `typing` performs caching of the outcome of parameterizing generics and `hash(True)` is equal to `hash(1)`. I think this behavior is incorrect and may lead to unexpected results.

Why is this inaccuracy important?
Consider the following example:

```python
from typing import Literal

SomeUsefulAlias = Literal[1, "abc"]

def func(arg: Literal[True, "abc"]):
    if arg is not True:
        arg.capitalize()
```

If we look at `func.__annotations__["arg"]`, we will see `Literal[1, 'abc']`. According to the new annotation, we can pass the value `1` to `func`, and this will lead to an attribute error, as you've already understood. Thus, proper runtime type checking cannot be performed.
msg405375 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-10-30 16:54
This happens to all `lru_cache` calls, where we deal with container types.

Example:

```
from functools import lru_cache

@lru_cache(typed=True)
def test(arg):
    return str(arg)

print(test((1, 'a')))  # (1, 'a')
print(test((True, 'a')))  # (1, 'a')
```

Moveover, this behavior is not tested in `test_functools`: https://github.com/python/cpython/blame/aae18a17401dc36917c0f64f971d60ab1a5b477e/Lib/test/test_functools.py#L1482-L1496

Original issue: https://bugs.python.org/issue13227
Original commit: https://github.com/python/cpython/commit/cd9fdfd652cdaca959b1c5d4cddf60d90a331b47
msg405376 - (view) Author: Nikita Sobolev (sobolevn) * (Python triager) Date: 2021-10-30 16:59
I see several ways of solving this:
1. Make `lru_cache(typed=True)` to look into iterable internals. This might be slow. And very slow for large iterables. Maybe even infinite for infinite ones
2. Adapt `Literal` / possibly other types to be treated differently. Because, most of the time, `Literal` take a small amount of args

I am going to send a PR with the second option.
msg405377 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-30 17:23
I agree. lru_cache(typed=True) itself should not look into iterable internals. It would be not only slow, but a change of semantic.

The simplest way to solve this issue is to remove caching of __getitem__(). The more sophisticated way is to move caching to lower level and apply it to a function with a var-positional parameter.
msg405394 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-31 08:22
New changeset 634984d7dbdd91e0a51a793eed4d870e139ae1e0 by Serhiy Storchaka in branch 'main':
bpo-45679: Fix caching of multi-value typing.Literal (GH-29334)
https://github.com/python/cpython/commit/634984d7dbdd91e0a51a793eed4d870e139ae1e0
msg405395 - (view) Author: miss-islington (miss-islington) Date: 2021-10-31 08:43
New changeset 3997f3ce8ab15269fc800062f75411865dbc0d55 by Miss Islington (bot) in branch '3.10':
bpo-45679: Fix caching of multi-value typing.Literal (GH-29334)
https://github.com/python/cpython/commit/3997f3ce8ab15269fc800062f75411865dbc0d55
msg405592 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-11-03 09:29
New changeset bbcf06bf95b448810f1b6f4f119c32e871bfb84c by Serhiy Storchaka in branch '3.9':
[3.9] bpo-45679: Fix caching of multi-value typing.Literal (GH-29334) (GH-29342)
https://github.com/python/cpython/commit/bbcf06bf95b448810f1b6f4f119c32e871bfb84c
msg405598 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-11-03 09:45
Closing this issue as the bug has been solved.

@Nikita could you please open a new issue for your tests PR and link to that instead? It seems like an enhancement over the current test suite for lru_cache. Thanks!
msg405825 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-11-05 19:52
New changeset 60b5333fa936a7e7f078a82e0fa3752cc9b6c5fb by Nikita Sobolev in branch 'main':
bpo-45679: add `tuple` tests with `lru_cache` to `test_functools` (GH-29339)
https://github.com/python/cpython/commit/60b5333fa936a7e7f078a82e0fa3752cc9b6c5fb
History
Date User Action Args
2022-04-11 14:59:51adminsetgithub: 89842
2021-11-05 19:52:54rhettingersetmessages: + msg405825
2021-11-03 09:45:01kjsetstatus: open -> closed
resolution: fixed
messages: + msg405598

stage: patch review -> resolved
2021-11-03 09:29:23kjsetmessages: + msg405592
2021-10-31 08:43:44miss-islingtonsetmessages: + msg405395
2021-10-31 08:41:44serhiy.storchakasetpull_requests: + pull_request27607
2021-10-31 08:22:26miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27606
2021-10-31 08:22:20serhiy.storchakasetmessages: + msg405394
2021-10-31 08:00:30sobolevnsetpull_requests: + pull_request27605
2021-10-30 18:17:58serhiy.storchakasetpull_requests: + pull_request27602
2021-10-30 17:29:04sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27601
2021-10-30 17:23:16serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg405377
2021-10-30 16:59:20sobolevnsetmessages: + msg405376
2021-10-30 16:54:44sobolevnsetnosy: + rhettinger, sobolevn, kj, gvanrossum
messages: + msg405375
2021-10-30 15:33:26vanburgerbergcreate