This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Make IndexError messages for list more informative
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.11
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Spencer Brown, aroberge, miguendes, mishra.akash.myself, pablogsal, remi.lapeyre, rhettinger, serhiy.storchaka, veky
Priority: normal Keywords: patch

Created on 2021-05-18 07:33 by miguendes, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26207 closed miguendes, 2021-05-18 07:40
Messages (11)
msg393858 - (view) Author: Miguel Brito (miguendes) * Date: 2021-05-18 07:33
I've noticed that in people will often ask for help on forums or stackoverflow to better understand the causes of
IndexErrors [1][2][3].
> The error message line for an IndexError doesn’t give you great information.[1]

Currently, when we access a out of bounds position we get something along the lines:
Traceback (most recent call last):

>>> a = [0, 10, 20, 30, 40]
>>> a[5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> a.pop(6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: pop index out of range
>>> a[6] = 7
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list assignment index out of range
>>> a = []
>>> a[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range


These error messages are too broad and doesn't inform the current length of the list, nor the index that was fed to it.

To improve the debugging experience in both interactive and non-interactive code,
I propose to offer a richer and more informative error messages. I think this can help users, especially newer and less
experienced folks, debug their code.


>>> a = [0, 10, 20, 30, 40]
>>> a[5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range, the len is 5 so index must be in -5..4, got 5
>>> a[-6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range, the len is 5 so index must be in -5..4, got -6
>>> a.pop(6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: pop index out of range, the len is 5 so index must be in -5..4, got 6
>>> a[6] = 7
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list assignment index out of range, the len is 5 so index must be in -5..4, got 6
>>> a = []
>>> a[2] = 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list assignment index out of range, got index 2 but the list is empty


These proposed messages are inspired by other languages, with the difference that in Python we can have negative index.
So informing the allowed ranges is a plus.

These improvements are not restricted to list, so it can be applied to strings and tuples as well, or any other indexable
object.

I have a branch ready, looking forward to hearing from you!

[1] https://stackoverflow.com/questions/1098643/indexerror-list-index-out-of-range-and-python
[2] https://stackoverflow.com/questions/16005707/index-error-list-index-out-of-range-python
[3] https://realpython.com/python-traceback/#indexerror
msg393861 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-05-18 09:10
The problem is that such exceptions are often used with the EAFP style. For example:

        while stop is None or i < stop:
            try:
                v = self[i]
                if v is value or v == value:
                    return i
            except IndexError:
                break
            i += 1

You caught IndexError and do something, but do not use the error message. Formatting error message takes a time. I am sure that error messages of a wast majority of IndexError exceptions are not used, just the type of the raised exception matters.
msg393866 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-05-18 09:50
I share the same concern as Serhiy, see my comment in the PR:

https://github.com/python/cpython/pull/26207#pullrequestreview-661799899
msg393905 - (view) Author: Miguel Brito (miguendes) * Date: 2021-05-18 20:26
Thanks for your comments, folks! I really appreciate it.

I left a comment about this issue in the PR thread: https://github.com/python/cpython/pull/26207#issuecomment-843531990
msg407707 - (view) Author: Akash Mishra (mishra.akash.myself) Date: 2021-12-05 12:29
While browsing some traceback error reporting i land on this solution threads. 

Specially beginner face traceback error for 
- KeyError
- IndexError (As mentioned in current message thread) 
- ValueError
..

It is sometime self explanatory from error reports but may be it can help to design some error document in python to more generalize errors.

Found some of traceback error with some possible cause explanations. Sharing if this is useful.  

https://github.com/pablorgarcia/Learn-to-program-with-Python-guide/blob/master/Understanding%20errors/IndexError-list-index-out-of-range.py

https://www.quizcure.com/python/traceback-most-recent-call-last-in-python#Keyerror-dictionary

https://stackoverflow.com/questions/64428188/deep-learning-chatbot-specific-index-error-list-index-out-of-range
msg407737 - (view) Author: Spencer Brown (Spencer Brown) * Date: 2021-12-05 20:35
One potential solution would be to add two Py_ssize_t to IndexError, storing the index and length along with the existing exception value. Then __str__() can append that to the message if set, perhaps having len be negative to signal they're not passed. 

An issue though is that it might be backwards incompatible with classes trying to multiply inherit from other exceptions too. To deal with that it could put the values in the args tuple, though then it'd have to allocate the tuple and ints which might hurt perf.
msg407744 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-05 21:17
I also share Serhiy's concerns and prefer that it be left as-is.

Conceptually, adding more information in the error message would make it more useful for debugging, but in practice, it would rarely be helpful.  In teaching and coaching Python, I've learned that the intended, dominant effect of seeing "IndexError: list index out of range" is that a person understands that index is invalid because it is out of range.  That is a helpful dominant impression.  IMO it would be distracting and rarely helpful to focus on the specific index and length.  The important point is that a nonsense lookup was being made.
msg407747 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-05 21:31
Here's another way to think of it:  

The call s[i] raising an IndexError isn't a numerical error, it is a conceptual error.  Knowing that i==15 and len(s)==10 doesn't usually help resolve the problem.  The fix typically isn't replacing s[i] with s[i - 5].  Usually the issue is usually deeper than that.

Note, this is different than KeyError where misspellings are common and where the fix can be indicated by knowing the key:   d['missteak'] -> d['mistake'].
msg407748 - (view) Author: Vedran Čačić (veky) * Date: 2021-12-05 21:55
> fix typically isn't replacing s[i] with s[i - 5]

... especially since that will still raise IndexError (in case when i==15 and len(s)==10). ;-P
msg407752 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-12-05 22:35
I concur with Serhiy and Raymond and I would prefer to leave as is.
msg407766 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-12-06 02:33
Thank for the suggestion.  While we're going to a pass on this one, no doubt there are other places that language can improve its communication with the user.  Please continue to submit ideas and patches.
History
Date User Action Args
2022-04-11 14:59:45adminsetgithub: 88332
2021-12-06 02:33:35rhettingersetmessages: + msg407766
2021-12-06 02:23:10rhettingersetassignee: rhettinger
2021-12-06 02:22:29rhettingersetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2021-12-05 22:35:22pablogsalsetmessages: + msg407752
2021-12-05 21:55:01vekysetnosy: + veky
messages: + msg407748
2021-12-05 21:31:17rhettingersetmessages: + msg407747
2021-12-05 21:17:35rhettingersetnosy: + rhettinger
messages: + msg407744
2021-12-05 20:35:28Spencer Brownsetnosy: + Spencer Brown
messages: + msg407737
2021-12-05 16:26:40arobergesetnosy: + aroberge
2021-12-05 12:29:53mishra.akash.myselfsetnosy: + mishra.akash.myself
messages: + msg407707
2021-05-20 10:18:06remi.lapeyresetnosy: + remi.lapeyre
2021-05-18 20:26:49miguendessetmessages: + msg393905
2021-05-18 09:50:58pablogsalsetmessages: + msg393866
2021-05-18 09:10:08serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg393861
2021-05-18 07:57:01xtreaksetnosy: + pablogsal
2021-05-18 07:40:49miguendessetkeywords: + patch
stage: patch review
pull_requests: + pull_request24824
2021-05-18 07:33:39miguendescreate