Title: Generic types accept indexing/subscripting, causing confusion
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.10, Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, gvanrossum, kj, levkivskyi, miss-islington, terry.reedy
Priority: normal Keywords: patch

Created on 2020-10-12 07:46 by kj, last changed 2020-10-21 00:01 by gvanrossum. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22667 closed kj, 2020-10-12 07:59
PR 22822 merged kj, 2020-10-20 16:49
PR 22840 merged miss-islington, 2020-10-20 23:38
Messages (9)
msg378475 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2020-10-12 07:46
While writing the docs for PEP 585's GenericAlias, I noticed that the following anti-pattern described in PEP 585 no longer throws an error when it should:

>>> l = list
>>> l[-1]

Whereas in versions of Python before 3.9, a "TypeError: 'type' object is not subscriptable" would have been thrown.

Quoting PEP 585 again: "Say, if a user is mistakenly passing a list type instead of a list object to a function, and that function is indexing the received object, the code would no longer raise an error."

Although the context of that statement isn't the same. I fully agree with its reasoning. This makes Python more confusing for beginners coming from other languages too:

// in c, makes a 10 element array 
int l[10];

# now in python, this is fine

This may give beginners the false impression that an empty 10 element list has been created.

I have created a PR and a test for this. The changed code blocks ints, floats, bools and complex from indexing for type objects. str is allowed since list['mytype'] seems analogous to typing.List[ForwardRef('mytype')]. Also, imo beginners are unlikely to accidentally write that code while intending to index a list.

Feedback is greatly appreciated, as I've not touched the C codebase in cpython before. Thanks!
msg378476 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2020-10-12 08:49
I'd like to clarify the following too, currently:

>>> list[1]

>>> list[1, 2, 3]
list[1, 2, 3]

The PR only solves (1) and not (2), is (2) intended behavior?
msg378533 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-10-12 23:22
Have you observed actual users getting confused by this? (E.g. StackOverflow or mailing list posts.) I would assume that if you try to *do* anything with the resulting object it's going to cause a traceback, and from there it's pretty straightforward to debug the situation. (Maybe not for absolute beginners, but they have a lot of ways to confuse themselves. :-)

There are people working on proposals for "integer variadics" that would actually give meaning to things like list[10] (or at least to A[10] and A[N, M] where A is a tensor type) so I am reluctant to forbid this now -- we might have to allow it in the next release.
msg378538 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2020-10-13 00:37
Partly due to Python 3.9.0's nascency, I've not been able to find online cases of beginners getting confused by this specific behavior.

Wow I didn't know about integer variadics, assuming they make their way into 3.11, I can see why it'd be better to leave that in for now.

I'm going to close the PR, and if there are no more discussions in the next 30 days, I'll close this bug report too. Thanks for clarifying.
msg378752 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-16 21:12
Subscription of list and dict (only with '') and ??? is at least puzzling.  Removal of a new feature after release is a bad idea.  But this new feature, expansion of subscription, needs to documented in
as legal code.

The current first line

 "A subscription selects an item of a sequence (string, tuple or list) or mapping (dictionary) object:"

is no longer always true.  This suggested replacement is:

 "Subscription of a sequence (string, tuple or list) or mapping (dictionary) object selects an item from the collection."

Then at the end, add something like

 "Subscription of certain type objects creates a Generic Alias.

where 'Generic Alias links to where such are documented.
msg378754 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-10-16 21:40
That sentence has been a lie for a long time (from the start?).

Any type can define __getitem__ and get this behavior, though in this case it is done through __class_getitem__ (PEP NNN).
msg379153 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-20 18:59
I think kj's patch is ready to merge.  Guido, do you want to take a look?
msg379173 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-10-20 23:38
New changeset 7cdf30fff39ea97f403b5472096349998d190e30 by kj in branch 'master':
bpo-42010: [docs] Clarify subscription of types (GH-22822)
msg379174 - (view) Author: miss-islington (miss-islington) Date: 2020-10-20 23:58
New changeset d05514ace652769457e6228f0186d4f830b676b9 by Miss Skeleton (bot) in branch '3.9':
[3.9] bpo-42010: [docs] Clarify subscription of types (GH-22822) (GH-22840)
Date User Action Args
2020-10-21 00:01:22gvanrossumsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-20 23:58:56miss-islingtonsetmessages: + msg379174
2020-10-20 23:38:34miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21794
2020-10-20 23:38:21gvanrossumsetmessages: + msg379173
2020-10-20 18:59:33terry.reedysetmessages: + msg379153
2020-10-20 16:49:00kjsetstage: needs patch -> patch review
pull_requests: + pull_request21779
2020-10-16 21:40:47gvanrossumsetmessages: + msg378754
2020-10-16 21:12:25terry.reedysetnosy: + terry.reedy, docs@python
messages: + msg378752

assignee: docs@python
components: + Documentation, - Interpreter Core
stage: patch review -> needs patch
2020-10-13 00:37:09kjsetmessages: + msg378538
2020-10-12 23:22:52gvanrossumsetmessages: + msg378533
2020-10-12 09:03:24serhiy.storchakasetnosy: + gvanrossum, levkivskyi
2020-10-12 08:49:53kjsetmessages: + msg378476
2020-10-12 08:07:53kjsettitle: Generic types accepts indexing/subscripting, causing confusion -> Generic types accept indexing/subscripting, causing confusion
2020-10-12 08:05:42kjsettitle: GenericAlias accepts indexing/subscripting, causing confusion -> Generic types accepts indexing/subscripting, causing confusion
2020-10-12 07:59:55kjsetkeywords: + patch
stage: patch review
pull_requests: + pull_request21640
2020-10-12 07:46:12kjcreate