classification
Title: Avoid raising OverflowError in len() when __len__() returns negative large value
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Oren Milman, barry, haypo, serhiy.storchaka, terry.reedy
Priority: normal Keywords:

Created on 2017-03-17 19:52 by serhiy.storchaka, last changed 2017-04-16 07:05 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 701 merged serhiy.storchaka, 2017-03-17 19:57
Messages (5)
msg289779 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-17 19:52
For now len() raises ValueError if __len__() returns small negative integer and OverflowError if __len__() returns large negative integer. 

>>> class NegativeLen:
...     def __len__(self):
...         return -10
... 
>>> len(NegativeLen())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: __len__() should return >= 0
>>> class HugeNegativeLen:
...     def __len__(self):
...         return -sys.maxsize-10
... 
>>> len(HugeNegativeLen())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer

Proposed patch makes it always raising ValueError.
msg289782 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2017-03-17 20:28
I was going to say that this is an API change, but given that without this, folks would have to catch both exceptions and now only have to catch one of them, it isn't.
msg290109 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-03-24 19:34
https://docs.python.org/3/library/functions.html#len * does not specify the exception.  In such cases, we occasionally change exception in x.y.0 releases without prior notice other than News and What's New.  Also, I think unnecessarily exposing a compile-switch dependent internal detail, as now, is almost a bug.  So +1 to applying in 3.7

* The doc does not specify that 'length' cannot be non-negative.  Perhaps it should, so no-one will think that they can hijack '__len__' to return something that is not a length, as usually understood.
msg291740 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 06:37
New changeset baf9f29811dba9c06e76b8e220bd77260202f299 by Serhiy Storchaka in branch 'master':
bpo-29839: Raise ValueError rather than OverflowError in len() for negative values. (#701)
https://github.com/python/cpython/commit/baf9f29811dba9c06e76b8e220bd77260202f299
msg291742 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-16 07:05
> The doc does not specify that 'length' cannot be non-negative.

It does. https://docs.python.org/3/reference/datamodel.html#object.__len__

.. method:: object.__len__(self)

   Called to implement the built-in function :func:`len`.  Should return the length
   of the object, an integer ``>=`` 0.
History
Date User Action Args
2017-04-16 07:05:36serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg291742

stage: patch review -> resolved
2017-04-16 06:37:20serhiy.storchakasetmessages: + msg291740
2017-04-07 15:54:02serhiy.storchakasetassignee: serhiy.storchaka
2017-03-24 19:34:42terry.reedysetnosy: + terry.reedy
messages: + msg290109
2017-03-17 20:35:34serhiy.storchakalinkissue29840 dependencies
2017-03-17 20:28:43barrysetnosy: + barry
messages: + msg289782
2017-03-17 19:58:01serhiy.storchakalinkissue29833 dependencies
2017-03-17 19:57:35serhiy.storchakasetpull_requests: + pull_request575
2017-03-17 19:52:14serhiy.storchakacreate