classification
Title: Consider reintroducing `types.EllipsisType` for the sake of typing
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BvB93, Mark.Shannon, eric.smith, gvanrossum, joshbode, levkivskyi, rhettinger, samuelmarks, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-09-18 19:29 by BvB93, last changed 2020-11-05 09:51 by eric.smith. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22336 merged BvB93, 2020-09-21 10:19
Messages (22)
msg377135 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-18 19:29
`Ellipsis` is one of the few builtin objects whose type is not exposed via the `types` module. 
This is not really an issue during runtime, as you can always call `type(Ellipsis)`, but for the purpose of typing it is detrimental; the lack of suitable type means that it is impossible to properly annotate a function which takes or returns `Ellipsis` (unless one is willing to resort to the use of non-public types: https://github.com/python/typeshed/issues/3556).

In order to resolve this issue I propose to reintroduce `types.EllipsisType`. This should be a fairly simple process, so if there are no objections I'd be willing to give it a shot.
msg377154 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-19 08:57
Can not `type(Ellipsis)` be used for typing too?
msg377165 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-19 13:32
If you're asking whether or not one can infer the return type of `type(Ellipsis)` then yes. 
In such case the inferred type is `builtins.ellipsis`, which is a private stub-only class (see the referenced typeshed issue in my original post).

If you're asking if a valid annotation can be constructed from `type(Ellipsis)` then the answer is unfortunately no (see below for a few examples).

```
EllipsisType = type(Ellipsis)

# Both examples are considered invalid
def func1(a: type(Ellipsis): ...
def func2(a: EllipsisType): ...

```
msg377167 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-19 14:23
Let’s do this.
msg377169 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-19 15:34
Would not be better to make MyPy supporting type(Ellipsis)? It would work also on Python versions older than 3.10.

Also, could not Literal[Ellipsis] be used as annotation?
msg377178 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-19 17:12
> Would not be better to make MyPy supporting type(Ellipsis)? It would work also on Python versions older than 3.10.

That would be quite complicated. There is no place in annotations where a function call is currently supported, so I'd prefer not to go there.

> Also, could not Literal[Ellipsis] be used as annotation?

Alas, it currently doesn't work (PEP 586 only allows specific types, and type checkers have implemented it exactly). Making it work would be more complicated than the proposal -- once it exists in types.py, it's trivial to add support to mypy.

IMO ideally, eventually, all "hidden" built-in types ought to be exposed somewhere, unless they are truly implementation details -- but since Ellipsis is a first-class singleton object, I don't see how its type could be an implementation detail. (Its name is actually clearly visible  in repr(type(Ellipsis)).)

Note that we have started exporting the types of other constructs through types.py, e.g. type(int|str) is types.Union, and type(list[str]) is types.GenericAlias. This is revealed in their repr().
msg377248 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-21 10:35
If we're going ahead with this: PR https://github.com/python/cpython/pull/22336 contains a concrete implementation of the proposed changes.
msg377250 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2020-09-21 11:10
I can't resist the pun on typing.
`type(...)` is the least typing :)

More seriously,

From a general consistency point of view,
if this is to be added, shouldn't `types.NoneType` and `types.NotImplementedType` be added as well?
msg377255 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-21 14:27
I’m okay with adding those (even though NoneType might not work in mypy, given the special-casing for None, per PEP 484).
msg377259 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-21 16:17
Apparently pyright has some interest in `NoneType` (https://github.com/python/typeshed/pull/4519), so it seems there are already some actual use cases.

In any case, I'm ok with adding `NoneType` and `NotImplementedType` to the PR.
msg377264 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-21 17:04
> In any case, I'm ok with adding `NoneType` and `NotImplementedType` to the PR.

Please do.
msg377267 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-21 18:37
`NoneType` and `NotImplementedType` have been added to the PR as of the latest set of pushes.
msg377269 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-09-21 19:03
Does anyone know why types.EllipsisType was removed to begin with? I just want to make sure we're not repeating some mistake of the past.
msg377274 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-21 19:30
Bas can you do some research to answer Eric's question? In theory it should be possible to use git bisect to find it. I see it in 2.7 but not in 3.4. That's a fairly large interval, alas.
msg377275 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-21 19:32
According to the relevant commit (https://github.com/python/cpython/commit/c9543e42330e5f339d6419eba6a8c5a61a39aeca):
"Removed all types from the 'types' module that are easily accessible through builtins."
msg377277 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-21 19:42
Thanks! Most of the deleted types *are* builtins (e.g. object, int). I assume the reasoning was just that if you wanted the type you could just write type(Ellipsis), type(None) etc.

But that's too dynamic for static checkers, so we want *some* stable name for those back. Do any of the other deleted types strike you as possibly needing to come back?
msg377280 - (view) Author: Bas van Beek (BvB93) * Date: 2020-09-21 19:59
> Do any of the other deleted types strike you as possibly needing to come back?

I don't think so, no. The only other one that stands out to me is `DictProxyType`, which has already been reintroduced as `MappingProxyType`.
msg377281 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-09-21 20:08
Thanks for doing the research, Bas! It sounds like adding back in NoneType, NotImplementedType, and EllipsisType is appropriate, then.

+1

The commit should have a comment about the reason: for type checkers which can't use type(Ellipsis), etc. I'll add a comment on the PR about adding a similar note to the blurb.
msg377282 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-09-21 20:57
> Do any of the other deleted types strike you as possibly
> needing to come back?

Yes.  NoneType would be useful.
msg377340 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-22 15:55
New changeset 0d0e9fe2ffc1683758a1985ef6dedeef5ecafdbc by Bas van Beek in branch 'master':
bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336)
https://github.com/python/cpython/commit/0d0e9fe2ffc1683758a1985ef6dedeef5ecafdbc
msg380400 - (view) Author: Samuel Marks (samuelmarks) * Date: 2020-11-05 09:44
Since we're bringing these back, would you accept a backport of these types?

[I'm writing a bunch of parsers/emitters—and starting to maintain an old runtime type-checker—for 3.6+]
msg380401 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-11-05 09:51
I don't think we should backport them. It's definitely a new feature, and our policy is no new features in micro versions.
History
Date User Action Args
2020-11-05 09:51:11eric.smithsetmessages: + msg380401
2020-11-05 09:44:14samuelmarkssetnosy: + samuelmarks
messages: + msg380400
2020-09-22 15:55:57gvanrossumsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-09-22 15:55:37gvanrossumsetmessages: + msg377340
2020-09-21 20:57:25rhettingersetnosy: + rhettinger
messages: + msg377282
2020-09-21 20:08:33eric.smithsetmessages: + msg377281
2020-09-21 19:59:23BvB93setmessages: + msg377280
2020-09-21 19:42:12gvanrossumsetmessages: + msg377277
2020-09-21 19:32:21BvB93setmessages: + msg377275
2020-09-21 19:30:15gvanrossumsetmessages: + msg377274
2020-09-21 19:03:23eric.smithsetnosy: + eric.smith
messages: + msg377269
2020-09-21 18:37:26BvB93setmessages: + msg377267
2020-09-21 17:04:52gvanrossumsetmessages: + msg377264
2020-09-21 16:17:16BvB93setmessages: + msg377259
2020-09-21 14:27:41gvanrossumsetmessages: + msg377255
2020-09-21 11:10:11Mark.Shannonsetnosy: + Mark.Shannon
messages: + msg377250
2020-09-21 10:35:29BvB93setmessages: + msg377248
2020-09-21 10:19:21BvB93setkeywords: + patch
stage: patch review
pull_requests: + pull_request21381
2020-09-19 17:12:30gvanrossumsetmessages: + msg377178
2020-09-19 15:34:28serhiy.storchakasetmessages: + msg377169
2020-09-19 14:23:05gvanrossumsetmessages: + msg377167
2020-09-19 13:32:55BvB93setmessages: + msg377165
2020-09-19 08:57:21serhiy.storchakasetnosy: + gvanrossum, serhiy.storchaka, levkivskyi
messages: + msg377154
2020-09-19 00:22:38joshbodesetnosy: + joshbode
2020-09-18 19:29:00BvB93create