msg391705 - (view) |
Author: FHTMitchell (FHTMitchell) |
Date: 2021-04-23 16:22 |
As of python 3.9, you now can't have multiple inheritance with `typing.NamedTuple` subclasses. This seems sensible, until you realise that `typing.Generic` works via inheritance. This fails whether or not `from __future__ import annotations` is enabled.
example:
```
class Group(NamedTuple, Generic[T]):
key: T
group: List[T]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-fc238c4826d7> in <module>
----> 1 class Group(NamedTuple, Generic[T]):
2 key: T
3 group: List[T]
4
~/.conda/envs/py39/lib/python3.9/typing.py in _namedtuple_mro_entries(bases)
1818 def _namedtuple_mro_entries(bases):
1819 if len(bases) > 1:
-> 1820 raise TypeError("Multiple inheritance with NamedTuple is not supported")
1821 assert bases[0] is NamedTuple
1822 return (_NamedTuple,)
TypeError: Multiple inheritance with NamedTuple is not supported
```
This worked fine in python 3.7 and 3.8 and as I understand it was one of the motivating cases for pep 560.
The change was made as part of bpo-40185: Refactor typing.NamedTuple. Whilst the obvious alternative is "use dataclasses", they don't have the same runtime properties or implications as namedtuples.
|
msg406083 - (view) |
Author: David Lukeš (dlukes) * |
Date: 2021-11-10 11:48 |
This is unfortunate, especially since it used to work... Going forward, is the intention not to support this use case? Or is it possible that support for generic NamedTuples will be re-added in the future?
|
msg406124 - (view) |
Author: Raymond Hettinger (rhettinger) * |
Date: 2021-11-10 18:57 |
+1 for reverting this change and restoring the previous behavior.
|
msg414541 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-04 17:54 |
Couldn't there be a subtler solution than rolling back GH-19371?
|
msg414557 - (view) |
Author: Jelle Zijlstra (JelleZijlstra) * |
Date: 2022-03-05 01:40 |
Was this ever documented to work? We have now disallowed this behavior in 3.9 and 3.10 with few complaints, so it doesn't seem that important to restore it. Also, static type checkers generally disallow generic NamedTuples.
|
msg414558 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-05 02:55 |
Mypy seems to allow this:
from typing import NamedTuple, TypeVar, Generic, List, Tuple
T = TypeVar("T")
class New(NamedTuple, Generic[T]):
x: List[T]
y: Tuple[T, T]
It's true that pyright doesn't, but maybe that's because it doesn't work in 3.9-3.10?
|
msg414559 - (view) |
Author: Jelle Zijlstra (JelleZijlstra) * |
Date: 2022-03-05 02:57 |
It doesn't really. If you do `x = New([1], (2, 3))` you get:
main.py:11: error: List item 0 has incompatible type "int"; expected "T"
main.py:11: error: Argument 2 to "New" has incompatible type "Tuple[int, int]"; expected "Tuple[T, T]"
https://mypy-play.net/?mypy=latest&python=3.10&gist=a13c7a33c55a3aeee95324d46cd03ffd
|
msg414563 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-05 05:12 |
So if it doesn't work in mypy why bother making it work at runtime? Is there an actual use case that broke? (There might be, but probably esoteric if nobody's run into it until now.)
|
msg414567 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-05 08:26 |
I actually have quite a few use cases for this feature. It's true that type checkers don't (yet) support it, but that doesn't mean that it should be disallowed at runtime. In fact, allowing it at runtime will surely give type checkers room to experiment with implementing this feature if it is requested by enough users. As it is, they are blocked from doing so.
|
msg414568 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-05 08:28 |
> Is there an actual use case that broke?
No, because this was never usable in the first place. But there are those who wish it were usable :)
|
msg414570 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2022-03-05 09:06 |
Please don't revert all changes. It will not make it working right in any case. See issue44863 for more correct approach.
|
msg414584 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-05 16:08 |
Can you be more specific about your use cases?
|
msg414585 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-05 16:49 |
Consider the typeshed stub for `concurrent.futures.DoneAndNotDoneFutures`. At runtime this is a `collections.namedtuple`, but in the stub, we need it to be generic to allow precise type inference. But we can't have a generic NamedTuple, so the stub is currently this:
```
class DoneAndNotDoneFutures(Sequence[set[Future[_T]]]):
@property
def done(self) -> set[Future[_T]]: ...
@property
def not_done(self) -> set[Future[_T]]: ...
def __new__(_cls, done: set[Future[_T]], not_done: set[Future[_T]]) -> DoneAndNotDoneFutures[_T]: ...
def __len__(self) -> int: ...
@overload
def __getitem__(self, __i: SupportsIndex) -> set[Future[_T]]: ...
@overload
def __getitem__(self, __s: slice) -> DoneAndNotDoneFutures[_T]: ...
```
Until two days ago, this stub actually had a bug: `done` and `not_done` were both given as writeable attributes, whereas they are read-only properties at runtime.
With generic NamedTuples, we could write the stub for the class far more simply (and more accurately) like this:
```
class DoneAndNotDoneFutures(NamedTuple, Generic[_T]):
done: set[Future[_T]]
not_done: set[Future[_T]]
```
And in code that actually needs to run at runtime, I frequently find it frustrating that I have to use dataclasses instead of NamedTuples if I want a simple class that just happens to be generic. dataclasses are great, but for small, lightweight classes, I prefer to use NamedTuples where possible. I often find that I don't need to use the full range of features dataclasses provide; and NamedTuples are often more performant than dataclasses, especially in cases where there's a lot of tuple unpacking.
|
msg414589 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-05 17:33 |
Okay, that's a sensible use case.
I do doubt your intuition of preferring named tuples over dataclasses a bit. This seems to encourage premature optimization. I'd say for simple cases use plain tuples (most performant), for complex cases use dataclasses (named fields and many other features that you may eventually want).
Compare concurent.futures.wait()'s return type (a named tuple) to asyncio.tasks.wait()'s return type (a plain tuple). I don't think that naming the fields of the return tuple (awkwardly :-) makes the c.f.wait() API easier to understand than the asyncio.wait() API.
Maybe named tuples, like typed dicts, are "in-between" solutions on the spectrum of data types (tuple - named tuple - dataclass; dict - typed dict - dataclass), and we should encourage people to use the neighboring solutions instead.
I'd rather spend efforts making dataclasses faster than adding features to named tuples.
|
msg414592 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-05 18:04 |
I sense we'll have to agree to disagree on the usefulness of NamedTuples in the age of dataclasses :)
For me, I find the simplicity of the underlying idea behind namedtuples — "tuples with some properties bolted on" — very attractive. Yes, standard tuples are more performant, but it's great to have a tool in the arsenal that's essentially the same as a tuple (and is backwards-compatible with a tuple, for APIs that require a tuple), but can also, like dataclasses, be self-documenting. (You're right that DoneAndNotDoneFutures isn't a great example of this.)
But I agree that this shouldn't be a priority if it's hard to accomplish; and there'll certainly be no complaints from me if energy is invested into making dataclasses faster.
|
msg414593 - (view) |
Author: Thomas Grainger (graingert) * |
Date: 2022-03-05 18:41 |
The main advantage for my usecase is support for heterogeneous unpacking
On Sat, Mar 5, 2022, 6:04 PM Alex Waygood <report@bugs.python.org> wrote:
>
> Alex Waygood <Alex.Waygood@Gmail.com> added the comment:
>
> I sense we'll have to agree to disagree on the usefulness of NamedTuples
> in the age of dataclasses :)
>
> For me, I find the simplicity of the underlying idea behind namedtuples —
> "tuples with some properties bolted on" — very attractive. Yes, standard
> tuples are more performant, but it's great to have a tool in the arsenal
> that's essentially the same as a tuple (and is backwards-compatible with a
> tuple, for APIs that require a tuple), but can also, like dataclasses, be
> self-documenting. (You're right that DoneAndNotDoneFutures isn't a great
> example of this.)
>
> But I agree that this shouldn't be a priority if it's hard to accomplish;
> and there'll certainly be no complaints from me if energy is invested into
> making dataclasses faster.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue43923>
> _______________________________________
>
|
msg414705 - (view) |
Author: Steven Silvester (Steven Silvester) |
Date: 2022-03-07 22:41 |
The use case that prompted https://github.com/python/cpython/pull/31679 is that we are adding typings to `PyMongo`. We are late to using typings, because we only recently dropped Python 2.7 support.
We have an existing options class that subclasses `NamedTuple`. We would like to make that class `Generic`, but are currently blocked.
Our current workaround is to create a separate stub file that uses `class CodecOptions(Tuple, Generic[T])` and explicitly re-declares the `NamedTuple` API.
Switching to `dataclass` would be disruptive, since we still support Python 3.6 and only rely on the standard library. We would also require a major version update since it would be an API change.
|
msg414709 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2022-03-08 00:37 |
Playing tricks where compile-time and run-time see slightly different types is probably more productive than trying to revert a PR that was in Python 3.9 and 3.10. :-)
I'm not opposed to supporting generic NamedTuple, but I expect the fix will never hit 3.9 and 3.10, and it needs to be a "fix forward" PR.
Would you mind closing the "revert" PR unmerged?
|
msg414754 - (view) |
Author: Steven Silvester (Steven Silvester) |
Date: 2022-03-08 14:02 |
I agree we're stuck with the typing stub workaround for our use case. We can re-submit a "fix forward" PR.
|
msg414791 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2022-03-09 10:21 |
PR 31781 is a simple PR which enables multiple inheritance with NamedTuple. As a side effect, it adds support of generic NamedTuple.
I am not sure that all details work as expected. It is easy to limit multiple inheritance only for Generic if needed.
|
msg414792 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-09 10:35 |
+1 for the more minimal changeset proposed in PR 31781. I've never felt a need for NamedTuple multiple inheritance other than with Generic, so wouldn't be opposed to restricting it only to Generic.
|
msg414815 - (view) |
Author: Julius Park (juliusgeo) * |
Date: 2022-03-09 21:20 |
What about Protocol? It is possible to create a dataclass that is a protocol, so it would be nicer from a symmetry perspective to allow it on both dataclasses and NamedTuples.
|
msg414816 - (view) |
Author: Jelle Zijlstra (JelleZijlstra) * |
Date: 2022-03-09 21:27 |
A NamedTuple that is also a Protocol doesn't make sense to me, since a NamedTuple is a concrete (nominal) type and a Protocol cannot inherit from a concrete type. If you want something like that to happen, it's better to open an issue on https://github.com/python/typing first, so this issue can stay focused on support for Generic + NamedTuple.
|
msg414817 - (view) |
Author: Alex Waygood (AlexWaygood) * |
Date: 2022-03-09 21:53 |
I agree with Jelle — a valid protocol cannot inherit from a concrete type, and the whole point of NamedTuple is that it creates a tuple subclass (and tuple is obviously a concrete type).
|
msg414830 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2022-03-10 08:17 |
What about mix-ins or abstract base classes?
class VecMixin:
def length(self):
return math.hypot(*self)
class Vec2D(NamedTuple, VecMixin):
x: float
y: float
class Vec3D(NamedTuple, VecMixin):
x: float
y: float
z: float
Currently you need to use the following trick to get a similar result:
class Vec2D(NamedTuple):
x: float
y: float
class Vec2D(Vec2D, VecMixin):
pass
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:44 | admin | set | github: 88089 |
2022-03-10 21:38:13 | behackett | set | nosy:
+ behackett
|
2022-03-10 08:17:42 | serhiy.storchaka | set | messages:
+ msg414830 |
2022-03-09 21:53:10 | AlexWaygood | set | messages:
+ msg414817 |
2022-03-09 21:27:06 | JelleZijlstra | set | messages:
+ msg414816 |
2022-03-09 21:20:50 | juliusgeo | set | messages:
+ msg414815 |
2022-03-09 10:35:58 | AlexWaygood | set | messages:
+ msg414792 |
2022-03-09 10:21:44 | serhiy.storchaka | set | messages:
+ msg414791 |
2022-03-09 10:16:07 | serhiy.storchaka | set | pull_requests:
+ pull_request29887 |
2022-03-09 04:16:10 | juliusgeo | set | nosy:
+ juliusgeo pull_requests:
+ pull_request29884
|
2022-03-08 14:02:35 | Steven Silvester | set | messages:
+ msg414754 |
2022-03-08 00:37:42 | gvanrossum | set | messages:
+ msg414709 |
2022-03-07 22:41:40 | Steven Silvester | set | nosy:
+ Steven Silvester messages:
+ msg414705
|
2022-03-05 18:41:34 | graingert | set | messages:
+ msg414593 |
2022-03-05 18:04:48 | AlexWaygood | set | messages:
+ msg414592 |
2022-03-05 17:33:51 | gvanrossum | set | messages:
+ msg414589 |
2022-03-05 16:51:41 | AlexWaygood | set | nosy:
+ graingert
|
2022-03-05 16:49:09 | AlexWaygood | set | messages:
+ msg414585 |
2022-03-05 16:08:04 | gvanrossum | set | messages:
+ msg414584 |
2022-03-05 09:06:46 | serhiy.storchaka | set | messages:
+ msg414570 |
2022-03-05 08:28:20 | AlexWaygood | set | messages:
+ msg414568 |
2022-03-05 08:26:53 | AlexWaygood | set | messages:
+ msg414567 |
2022-03-05 05:12:39 | gvanrossum | set | messages:
+ msg414563 |
2022-03-05 02:57:54 | JelleZijlstra | set | messages:
+ msg414559 |
2022-03-05 02:55:26 | gvanrossum | set | messages:
+ msg414558 |
2022-03-05 01:40:45 | JelleZijlstra | set | messages:
+ msg414557 |
2022-03-04 17:54:39 | gvanrossum | set | messages:
+ msg414541 |
2022-03-04 10:20:15 | AlexWaygood | set | nosy:
+ gvanrossum, JelleZijlstra, sobolevn, kj
|
2022-03-04 09:40:03 | python-dev | set | keywords:
+ patch nosy:
+ python-dev
pull_requests:
+ pull_request29798 stage: patch review |
2021-11-10 23:15:18 | AlexWaygood | set | nosy:
+ AlexWaygood
|
2021-11-10 18:57:26 | rhettinger | set | messages:
+ msg406124 |
2021-11-10 11:48:28 | dlukes | set | nosy:
+ dlukes messages:
+ msg406083
|
2021-04-23 19:12:50 | eric.smith | set | nosy:
+ eric.smith
|
2021-04-23 16:27:00 | rhettinger | set | nosy:
+ rhettinger, serhiy.storchaka, levkivskyi
|
2021-04-23 16:23:07 | FHTMitchell | set | type: behavior |
2021-04-23 16:22:56 | FHTMitchell | create | |