classification
Title: PEP 604 Union (int | str) doesn't have __parameters__
Type: enhancement Stage: resolved
Components: Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Jelle Zijlstra, ROpdebee, brandtbucher, gvanrossum, jack__d, kj, levkivskyi, lukasz.langa, miss-islington, serhiy.storchaka, uriyyo
Priority: normal Keywords: patch

Created on 2021-06-22 13:29 by kj, last changed 2021-07-26 19:32 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26980 merged uriyyo, 2021-07-01 09:04
PR 27048 merged uriyyo, 2021-07-06 19:26
PR 27203 closed kj, 2021-07-17 03:50
PR 27207 merged serhiy.storchaka, 2021-07-17 06:49
PR 27215 merged uriyyo, 2021-07-17 10:23
PR 27220 closed miss-islington, 2021-07-18 02:27
PR 27222 merged kj, 2021-07-18 07:18
PR 27368 merged miss-islington, 2021-07-26 16:06
Messages (19)
msg396329 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-06-22 13:29
Recently I noticed that the new PEP 604 Union type doesn't collect type variables:

from typing import TypeVar
T = TypeVar('T')

(int | list[T]).__parameters__

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'types.Union' object has no attribute '__parameters__'

Whereas the typing.Union version has __parameters__. Is this behavior intentional?

The downside to this is that things like this don't work:

alias: TypeAlias = int | list[T]
alias[str] # Error!
msg396346 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2021-06-22 15:00
I agree that this is a bug. `types.Union` is also missing a __getitem__ implementation.

And `typing.Union` supports pickling while `types.Union` doesn't:

>>> pickle.loads(pickle.dumps(int | str))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot pickle 'types.Union' object
>>> pickle.loads(pickle.dumps(Union[int, str]))
typing.Union[int, str]

I don't have a use case for pickling types but someone might.
msg396351 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-06-22 15:55
Let's first see whether the type (int | list[T]) is accepted by static checkers.

IMO introspecting unions should be done by looking at __args__, nothing more.
msg396709 - (view) Author: Yurii Karabas (uriyyo) * (Python triager) Date: 2021-06-29 10:00
Should `__getitem__` be implemented for `types.Union`?
I can implement it if no one is working on it.

P.S. mypy currently does not support it:
```
Value of type "types.Union" is not indexable
```
msg396710 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-06-29 10:21
Yurii, thanks for the offer.

We only need to implement __getitem__ if union supports TypeVars. Which
means __parameters__ need to be implemented too (or at least a private
internal implementation of it).

I interpreted Guido's message above as to wait and see if static type
checkers even accept things like int | list[T]. Maybe he meant that if that
syntax isnt valid then theres no point for us to implement it? PEP 604
leaves out how it deals with TypeVars, so adding this behavior may require
us to update the PEP first (which will require approval from others).

Anyways, there's no rush. We probably can't backport this so we have until
the next minor release of Python (3.11 or later) to decide.
msg396755 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-06-29 20:40
I intended for someone to write some test programs and report back here what mypy actually supports and what it doesn't.
msg396756 - (view) Author: Jack DeVries (jack__d) * Date: 2021-06-29 22:16
mypy does not support __parameters__:

    (venv) ➜  cpython git:(main) cat repro.py 
    from typing import TypeVar
    T = TypeVar('T')

    (int | list[T]).__parameters__
    (venv) ➜  cpython git:(main) mypy --version
    mypy 0.920+dev.cae5d3c8b5f14d0796914aa6a113473ca3ffc38e
    (venv) ➜  cpython git:(main) python --version
    Python 3.11.0a0
    (venv) ➜  cpython git:(main) mypy repro.py 
    repro.py:4: error: "types.Union" has no attribute "__parameters__"
    Found 1 error in 1 file (checked 1 source file)
    (venv) ➜  cpython git:(main)


mypy also does not support __getitem__

    (venv) ➜  cpython git:(main) cat repro.py 
    from typing import TypeVar
    T = TypeVar('T')

    (int | list[T]).__getitem__
    (venv) ➜  cpython git:(main) mypy --version
    ./mypy 0.920+dev.cae5d3c8b5f14d0796914aa6a113473ca3ffc38e
    (venv) ➜  cpython git:(main) python --version
    Python 3.11.0a0
    (venv) ➜  cpython git:(main) mypy repro.py 
    repro.py:4: error: Value of type "types.Union" is not indexable
    Found 1 error in 1 file (checked 1 source file)
    (venv) ➜  cpython git:(main)
msg396758 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2021-06-29 22:26
Mypy is definitely not going to support direct access to `__parameters__`; what Guido is referring to is whether usage of types.Union that would require `__parameters__` at runtime is accepted by mypy.

For example, this:

from typing import TypeVar

T = TypeVar("T")

Alias = int | list[T]

def f(x: Alias[str]) -> None:
    pass

But this produces `main.py:7: error: Variable "main.Alias" is not valid as a type`: mypy doesn't even recognize `|` as a type yet.

I'd rather not focus too much though on what mypy does; it is one of many type checkers by now and does not tend to be the quickest in adding support for new features.

Pyright does handle the file above as I'd expect, for what it's worth.
msg396759 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-06-29 22:41
So I guess to expand on Jelle's example, Alias[str] should return (int |
list[str]), right? That makes sense since that's how Union works.
msg396760 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * (Python triager) Date: 2021-06-29 22:43
I'd also be OK with returning a `types.GenericAlias(int | list[T], str)`, which might be simpler. It doesn't matter for static type checkers, and runtime type checkers can extract what they need anyway.
msg396767 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-06-29 23:57
I don't think we need the types.GenericAlias(int | list[T], str)
workaround. GenericAlias already has code for extracting __parameters__ and
implementing __getitem__. Someone would need to refactor and reuse them for
Union too. That should be enough to trick GenericAlias to treat Union as
another GenericAlias(I don't remember if it checks for __origin__ as well)
and cover other edge cases as well.

@Yurii do you still plan to take this? If not, I'll start working on
something later today.
msg396770 - (view) Author: Yurii Karabas (uriyyo) * (Python triager) Date: 2021-06-30 08:02
@Ken I will pick up this issue, thanks for asking.
msg396895 - (view) Author: Ruben Opdebeeck (ROpdebee) Date: 2021-07-03 11:13
It also lacks the __module__ attribute, causing it to be unusable in PEP 593 typing.Annotated types:

from typing import Annotated
x: Annotated[int | str, 'test']

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 298, in inner
    return cached(*args, **kwds)
  File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 1594, in __class_getitem__
    return _AnnotatedAlias(origin, metadata)
  File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 1520, in __init__
    super().__init__(origin, origin)
  File "/Users/ruben/.pyenv/versions/3.10.0b3/lib/python3.10/typing.py", line 976, in __init__
    self.__module__ = origin.__module__
AttributeError: 'types.Union' object has no attribute '__module__'. Did you mean: '__reduce__'?
msg397051 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-06 18:04
New changeset c45fa1a5d9b419cf13ad4b5a7cb453956495b83e by Yurii Karabas in branch 'main':
bpo-44490: Add __parameters__ and __getitem__ to types.Union (GH-26980)
https://github.com/python/cpython/commit/c45fa1a5d9b419cf13ad4b5a7cb453956495b83e
msg397686 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-17 03:33
New changeset bf89ff96e6ba21bb52b8597b5e51e8ffc57e6589 by Yurii Karabas in branch 'main':
bpo-44490: Improve typing module compatibility with types.Union (GH-27048)
https://github.com/python/cpython/commit/bf89ff96e6ba21bb52b8597b5e51e8ffc57e6589
msg397728 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-17 19:15
New changeset 2d055ce13250a4074f66a945381a149a3cf8c46f by Serhiy Storchaka in branch '3.10':
[3.10] bpo-44490: Add __parameters__ and __getitem__ to types.Union (GH-26980) (GH-27207)
https://github.com/python/cpython/commit/2d055ce13250a4074f66a945381a149a3cf8c46f
msg397801 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-19 14:23
New changeset a2721649598eb715304a1ac8678a409585f73b27 by Ken Jin in branch '3.10':
bpo-44490: Improve typing module compatibility with types.Union (GH-27048) (#27222)
https://github.com/python/cpython/commit/a2721649598eb715304a1ac8678a409585f73b27
msg398241 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-26 16:06
New changeset 6c1b57d958e2b0d5bcf0f2315b5044838e11638f by Yurii Karabas in branch 'main':
bpo-44490: Add 'Whats New' docs regarding types.Union changes (GH-27215)
https://github.com/python/cpython/commit/6c1b57d958e2b0d5bcf0f2315b5044838e11638f
msg398251 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-26 19:32
New changeset 4a5457d5d48e6563ec481daad245ff3d6ef13503 by Miss Islington (bot) in branch '3.10':
bpo-44490: Add 'Whats New' docs regarding types.Union changes (GH-27215) (GH-27368)
https://github.com/python/cpython/commit/4a5457d5d48e6563ec481daad245ff3d6ef13503
History
Date User Action Args
2021-07-26 19:32:08lukasz.langasetmessages: + msg398251
2021-07-26 16:06:04lukasz.langasetnosy: + lukasz.langa
messages: + msg398241
2021-07-26 16:06:04miss-islingtonsetpull_requests: + pull_request25906
2021-07-19 14:23:25gvanrossumsetmessages: + msg397801
2021-07-18 07:18:22kjsetpull_requests: + pull_request25763
2021-07-18 04:02:11kjsetpull_requests: - pull_request25752
2021-07-18 02:27:52miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request25761
2021-07-17 19:15:26serhiy.storchakasetmessages: + msg397728
2021-07-17 10:23:02uriyyosetpull_requests: + pull_request25753
2021-07-17 10:01:56uriyyosetpull_requests: + pull_request25752
2021-07-17 06:49:12serhiy.storchakasetnosy: + serhiy.storchaka

pull_requests: + pull_request25743
2021-07-17 03:50:42kjsetpull_requests: + pull_request25737
2021-07-17 03:45:14gvanrossumsetstatus: open -> closed
type: enhancement
resolution: fixed
stage: patch review -> resolved
2021-07-17 03:33:44gvanrossumsetmessages: + msg397686
2021-07-06 19:26:53uriyyosetpull_requests: + pull_request25604
2021-07-06 18:04:58gvanrossumsetmessages: + msg397051
2021-07-03 11:13:17ROpdebeesetnosy: + ROpdebee
messages: + msg396895
2021-07-01 09:04:09uriyyosetkeywords: + patch
stage: patch review
pull_requests: + pull_request25542
2021-06-30 08:02:45uriyyosetmessages: + msg396770
2021-06-29 23:57:06kjsetmessages: + msg396767
2021-06-29 22:43:44Jelle Zijlstrasetmessages: + msg396760
2021-06-29 22:41:43gvanrossumsetmessages: + msg396759
2021-06-29 22:26:10Jelle Zijlstrasetmessages: + msg396758
2021-06-29 22:16:44jack__dsetnosy: + jack__d
messages: + msg396756
2021-06-29 20:40:27gvanrossumsetmessages: + msg396755
2021-06-29 10:21:05kjsetmessages: + msg396710
2021-06-29 10:00:47uriyyosetnosy: + uriyyo
messages: + msg396709
2021-06-23 18:28:56brandtbuchersetnosy: + brandtbucher
2021-06-22 15:55:18gvanrossumsetmessages: + msg396351
2021-06-22 15:00:20Jelle Zijlstrasetmessages: + msg396346
2021-06-22 13:29:59kjcreate