This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: types.UnionType is not subscriptable
Type: Stage: resolved
Components: Versions: Python 3.10
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, joperez, kj, serhiy.storchaka
Priority: normal Keywords:

Created on 2021-10-09 21:37 by joperez, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (8)
msg403554 - (view) Author: Joseph Perez (joperez) * Date: 2021-10-09 21:37
`types.UnionType` is not subscriptable, and this is an issue when type manipulations are done.

A common maniputation I've to do is to substitute all the `TypeVar` of a potential generic type by their specialization in the given context.
For example, given a class:
```python
@dataclass
class Foo(Generic[T]):
    bar: list[T]
    baz: T | None
```
in the case of `Foo[int]`, I want to compute the effective type of the fields, which will be `list[int]` and `int | None`.
It could be done pretty easily by a recursive function:
```python
def substitute(tp, type_vars: dict):
    origin, args = get_origin(tp), get_args(tp)
    if isinstance(tp, TypeVar):
        return type_vars.get(tp, tp)
    elif origin is Annotated:
        return Annotated[(substitute(args[0], type_vars), *args[1:])]
    else:
        return origin[tuple(substitute(arg) for arg in args)]  # this line fails for types.UnionType
```

And this is not the only manipulation I've to do on generic types. In fact, all my library (apischema) is broken in Python 3.10 because of `types.UnionType`.
I've to substitute `types.UnionType` by `typing.Union` everywhere to make things work; `types.UnionType` is just not usable for dynamic manipulations.

I've read PEP 604 and it doesn't mention if `types.UnionType` should be subscriptable or not. Is there a reason for not making it subscriptable?
msg403573 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-10-10 05:59
I don't understand your example, T | None doesn't return a types.Union object, it returns typing.Union/typing.Optional. (I'm assuming this T is the TypeVar in typing). Which *is* subscriptable.

>>> (T | None)[int].__origin__
typing.Union

If you meant to say: why is typing.Union[] allowed, but not types.UnionType[]? That is intentional. types.UnionType is only meant for builtin types. Once you union with *any* type from typing, it will convert to a typing.Union.

>>> type(int | str)
<class 'types.UnionType'>

>>> int | str | T
typing.Union[int, str, ~T]

If you intend to reconstruct a types.Union from another types.Union, you can do:

args = get_args(int | str)
import operator, functools
functools.reduce(operator.or_, args)

And guard this code with an isinstance(tp, types.UnionType) check.
msg403575 - (view) Author: Joseph Perez (joperez) * Date: 2021-10-10 07:24
Indeed, sorry, my example was bad. My library was raising at several place, and I've extrapolated about generic substitution.

I've indeed other substitutions (without `TypeVar`), and because they were failing, I've assumed that all of my substitutions were failing; I was wrong about generic one.

For example, if I want to substitute `int | Collection[int]` to `int | list[int]`, I will have to replace `types.UnionType` by `typing.Union` or use `reduce`, while it was not necessary in 3.9 where I could just write `get_origin(tp)[new_args]`.

So I'll have to add some `if` in my code.
msg403576 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-10-10 07:50
No worries!

> So I'll have to add some `if` in my code.

Yeah, we had to do that in the typing module too. Hope you manage to fix your library without much trouble.
msg403594 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-10-10 15:16
> If you meant to say: why is typing.Union[] allowed, but not
types.UnionType[]? That is intentional. types.UnionType is only meant for
builtin types. Once you union with *any* type from typing, it will convert
to a typing.Union.

But why? Just so types.UnionType (if it has a typevar) doesn’t have to
support subscriptions? Even if this saves us now, I agree with OP that it
ought to allow it, so we can deprecate typing.Union properly. And e.g.
dict[str, T] works.
-- 
--Guido (mobile)
msg403596 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-10-10 15:38
@Guido,

I hope I didn't misunderstand you, but to clarify, what OP is asking is an alternative way to construct types.UnionType objects and write:

types.UnionType[int, str]

like how we used to write before 3.10:

typing.Union[int, str]

I don't know why we need this. We can write `int | str`. The reason for PEP 604 in the first place was to avoid the subscript syntax and use `|` since it's cleaner. OP's use case is for reconstructing types.UnionType objects easily, but `functools.reduce(operator.or_, args)` works.

Re: TypeVar subscription; PEP 604 syntax already supports that. We used to implement that in C. After Serhiy's Great Cleanup, a bitwise OR with a TypeVar automatically converts types.UnionType to typing.Union. So all the TypeVar support is now done in Python.

>>> type(int | str)
<class 'types.UnionType'>

>>> (int | str | T)[dict]
typing.Union[int, str, dict]
msg403597 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-10-10 15:53
Oh, I see. No, they must make another special case, like for Annotated.--
--Guido (mobile)
msg403605 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-10-10 18:36
types.UnionType corresponds to typing._UnionGenericAlias, not typing.Union.

We can make (int | str | T)[dict] returning an instance of types.UnionType instead of an instance of typing._UnionGenericAlias. But it will be a breaking change, because typing._UnionGenericAlias and types.UnionType are different and not completely compatible types. We should wait some time before making such changes, so all user code will be made supporting both typing._UnionGenericAlias and types.UnionType.

If the user code does something special like substituting `int | Collection[int]` to `int | list[int]`, it should have some additional ifs in any case, otherwise it will not recognize new typing types including types.UnionTypes. And subscription does not work in all typing types, we have copy_with() for some types and special cases for others in the code of the typing module. I am going to unify it finally, but it takes time, my time and user's time to migrate to new idioms.
History
Date User Action Args
2022-04-11 14:59:51adminsetgithub: 89581
2021-10-10 18:36:04serhiy.storchakasetmessages: + msg403605
2021-10-10 15:53:01gvanrossumsetmessages: + msg403597
2021-10-10 15:38:27kjsetmessages: + msg403596
2021-10-10 15:16:50gvanrossumsetmessages: + msg403594
2021-10-10 07:50:23kjsetmessages: + msg403576
2021-10-10 07:24:44joperezsetstatus: pending -> closed

messages: + msg403575
stage: resolved
2021-10-10 05:59:28kjsetstatus: open -> pending
nosy: + gvanrossum, kj, serhiy.storchaka
messages: + msg403573

2021-10-09 21:37:56joperezcreate