Issue45418
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-10-09 21:37 by joperez, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (8) | |||
---|---|---|---|
msg403554 - (view) | Author: Joseph Perez (joperez) * | Date: 2021-10-09 21:37 | |
`types.UnionType` is not subscriptable, and this is an issue when type manipulations are done. A common maniputation I've to do is to substitute all the `TypeVar` of a potential generic type by their specialization in the given context. For example, given a class: ```python @dataclass class Foo(Generic[T]): bar: list[T] baz: T | None ``` in the case of `Foo[int]`, I want to compute the effective type of the fields, which will be `list[int]` and `int | None`. It could be done pretty easily by a recursive function: ```python def substitute(tp, type_vars: dict): origin, args = get_origin(tp), get_args(tp) if isinstance(tp, TypeVar): return type_vars.get(tp, tp) elif origin is Annotated: return Annotated[(substitute(args[0], type_vars), *args[1:])] else: return origin[tuple(substitute(arg) for arg in args)] # this line fails for types.UnionType ``` And this is not the only manipulation I've to do on generic types. In fact, all my library (apischema) is broken in Python 3.10 because of `types.UnionType`. I've to substitute `types.UnionType` by `typing.Union` everywhere to make things work; `types.UnionType` is just not usable for dynamic manipulations. I've read PEP 604 and it doesn't mention if `types.UnionType` should be subscriptable or not. Is there a reason for not making it subscriptable? |
|||
msg403573 - (view) | Author: Ken Jin (kj) * | Date: 2021-10-10 05:59 | |
I don't understand your example, T | None doesn't return a types.Union object, it returns typing.Union/typing.Optional. (I'm assuming this T is the TypeVar in typing). Which *is* subscriptable. >>> (T | None)[int].__origin__ typing.Union If you meant to say: why is typing.Union[] allowed, but not types.UnionType[]? That is intentional. types.UnionType is only meant for builtin types. Once you union with *any* type from typing, it will convert to a typing.Union. >>> type(int | str) <class 'types.UnionType'> >>> int | str | T typing.Union[int, str, ~T] If you intend to reconstruct a types.Union from another types.Union, you can do: args = get_args(int | str) import operator, functools functools.reduce(operator.or_, args) And guard this code with an isinstance(tp, types.UnionType) check. |
|||
msg403575 - (view) | Author: Joseph Perez (joperez) * | Date: 2021-10-10 07:24 | |
Indeed, sorry, my example was bad. My library was raising at several place, and I've extrapolated about generic substitution. I've indeed other substitutions (without `TypeVar`), and because they were failing, I've assumed that all of my substitutions were failing; I was wrong about generic one. For example, if I want to substitute `int | Collection[int]` to `int | list[int]`, I will have to replace `types.UnionType` by `typing.Union` or use `reduce`, while it was not necessary in 3.9 where I could just write `get_origin(tp)[new_args]`. So I'll have to add some `if` in my code. |
|||
msg403576 - (view) | Author: Ken Jin (kj) * | Date: 2021-10-10 07:50 | |
No worries! > So I'll have to add some `if` in my code. Yeah, we had to do that in the typing module too. Hope you manage to fix your library without much trouble. |
|||
msg403594 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2021-10-10 15:16 | |
> If you meant to say: why is typing.Union[] allowed, but not types.UnionType[]? That is intentional. types.UnionType is only meant for builtin types. Once you union with *any* type from typing, it will convert to a typing.Union. But why? Just so types.UnionType (if it has a typevar) doesn’t have to support subscriptions? Even if this saves us now, I agree with OP that it ought to allow it, so we can deprecate typing.Union properly. And e.g. dict[str, T] works. -- --Guido (mobile) |
|||
msg403596 - (view) | Author: Ken Jin (kj) * | Date: 2021-10-10 15:38 | |
@Guido, I hope I didn't misunderstand you, but to clarify, what OP is asking is an alternative way to construct types.UnionType objects and write: types.UnionType[int, str] like how we used to write before 3.10: typing.Union[int, str] I don't know why we need this. We can write `int | str`. The reason for PEP 604 in the first place was to avoid the subscript syntax and use `|` since it's cleaner. OP's use case is for reconstructing types.UnionType objects easily, but `functools.reduce(operator.or_, args)` works. Re: TypeVar subscription; PEP 604 syntax already supports that. We used to implement that in C. After Serhiy's Great Cleanup, a bitwise OR with a TypeVar automatically converts types.UnionType to typing.Union. So all the TypeVar support is now done in Python. >>> type(int | str) <class 'types.UnionType'> >>> (int | str | T)[dict] typing.Union[int, str, dict] |
|||
msg403597 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2021-10-10 15:53 | |
Oh, I see. No, they must make another special case, like for Annotated.-- --Guido (mobile) |
|||
msg403605 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2021-10-10 18:36 | |
types.UnionType corresponds to typing._UnionGenericAlias, not typing.Union. We can make (int | str | T)[dict] returning an instance of types.UnionType instead of an instance of typing._UnionGenericAlias. But it will be a breaking change, because typing._UnionGenericAlias and types.UnionType are different and not completely compatible types. We should wait some time before making such changes, so all user code will be made supporting both typing._UnionGenericAlias and types.UnionType. If the user code does something special like substituting `int | Collection[int]` to `int | list[int]`, it should have some additional ifs in any case, otherwise it will not recognize new typing types including types.UnionTypes. And subscription does not work in all typing types, we have copy_with() for some types and special cases for others in the code of the typing module. I am going to unify it finally, but it takes time, my time and user's time to migrate to new idioms. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:51 | admin | set | github: 89581 |
2021-10-10 18:36:04 | serhiy.storchaka | set | messages: + msg403605 |
2021-10-10 15:53:01 | gvanrossum | set | messages: + msg403597 |
2021-10-10 15:38:27 | kj | set | messages: + msg403596 |
2021-10-10 15:16:50 | gvanrossum | set | messages: + msg403594 |
2021-10-10 07:50:23 | kj | set | messages: + msg403576 |
2021-10-10 07:24:44 | joperez | set | status: pending -> closed messages: + msg403575 stage: resolved |
2021-10-10 05:59:28 | kj | set | status: open -> pending nosy: + gvanrossum, kj, serhiy.storchaka messages: + msg403573 |
2021-10-09 21:37:56 | joperez | create |