classification
Title: Possible OverflowError in __len__ method undocumented (when called via len() function)
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, Rostyslav.Dzinko, Winterflower, benjamin.peterson, docs@python, georg.brandl, martin.panter, serhiy.storchaka, terry.reedy, vstinner
Priority: normal Keywords:

Created on 2012-08-17 11:17 by Rostyslav.Dzinko, last changed 2020-01-10 12:13 by Zac Hatfield-Dodds. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1256 merged serhiy.storchaka, 2017-04-22 15:08
PR 1259 merged serhiy.storchaka, 2017-04-23 05:42
PR 1260 merged serhiy.storchaka, 2017-04-23 05:43
PR 1261 merged serhiy.storchaka, 2017-04-23 05:49
PR 17934 merged Zac Hatfield-Dodds, 2020-01-10 12:13
Messages (10)
msg168439 - (view) Author: Rostyslav Dzinko (Rostyslav.Dzinko) Date: 2012-08-17 11:17
I've encountered that OverflowError which can happen in __len__ method is still undocumented, though one issue on this problem: http://bugs.python.org/issue12159 ended up with "need to be documented" comment.

Link to documentation: http://docs.python.org/reference/datamodel.html#object.__len__

I think it must be clarified that __len__ return value is constrained to upper boundary (Py_ssize_t c type) when you plan to call it via len() built-in function.
msg168473 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-08-17 20:18
In #12159, Victor correctly labelled this an implementation (IE, CPython) detail (limitation). I don't believe any implementation has to limit the range of len(). So the question is whether we should add a CPython implementation limit note, including the possibility of OverflowError, and if so, to both len() and __len__() entries. I am not sure of the current doc policy.

The second sentence of the len entry is out of date. " The argument may be a sequence (string, tuple or list) or a mapping (dictionary)." Sets and any collections with a size (__len__ method) can also be arguments. I am not sure how to revise that either.
msg257312 - (view) Author: Camilla Montonen (Winterflower) Date: 2016-01-01 22:31
The deficiency noticed by Terry has been at least partially corrected in the len() docs
https://docs.python.org/2/library/functions.html#len
Return the length (the number of items) of an object. The argument may be a sequence (such as a string, bytes, tuple, list, or range) or a collection (such as a dictionary, set, or frozen set).

It doesn't mention the fact that the collection has to implement __len__, but I suppose that might be obvious to experienced Python users (isn't obvious to a beginner like me, though). So perhaps this might be a welcome clarification. 

The issue regarding OverflowError raised by Rostyslav still remains unresolved.
msg257318 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-01 23:47
Issue 10289 proposes to link from len() to __len__() in the documentation. I think the limitation only needs to be documented for __len__(); there are other ways to invoke it as well.
msg292123 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-04-22 16:09
Has or will builtin classes be modified to follow advice to include a separate __bool__?  In 3.6.1,

>>> r = range(19**100)
>>> bool(r)
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    bool(r)
OverflowError: Python int too large to convert to C ssize_t
msg292125 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-22 16:28
Yes, and the range object already is fixed by issue28876.
msg292151 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-23 05:38
New changeset 85157cd89a6edac347a5b6871fcf20c500c6fbbf by Serhiy Storchaka in branch 'master':
bpo-15718: Document the upper bound constrain on the __len__ return value. (#1256)
https://github.com/python/cpython/commit/85157cd89a6edac347a5b6871fcf20c500c6fbbf
msg292152 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-23 05:50
New changeset f2ed2858eea7569c8915b3611ca5ec92ae10b17f by Serhiy Storchaka in branch '3.6':
[3.6] bpo-15718: Document the upper bound constrain on the __len__ return value. (GH-1256) (#1259)
https://github.com/python/cpython/commit/f2ed2858eea7569c8915b3611ca5ec92ae10b17f
msg292153 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-23 05:50
New changeset c358536fd5e40e8f29ee4f086588a82fccb25a09 by Serhiy Storchaka in branch '3.5':
[3.5] bpo-15718: Document the upper bound constrain on the __len__ return value. (GH-1256) (#1260)
https://github.com/python/cpython/commit/c358536fd5e40e8f29ee4f086588a82fccb25a09
msg292154 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-23 05:58
New changeset bfc7dff63b9b30371e3423a5c35ccda2f3b52218 by Serhiy Storchaka in branch '2.7':
[2.7] bpo-15718: Document the upper bound constrain on the __len__ return value. (GH-1256). (#1261)
https://github.com/python/cpython/commit/bfc7dff63b9b30371e3423a5c35ccda2f3b52218
History
Date User Action Args
2020-01-10 12:13:55Zac Hatfield-Doddssetpull_requests: + pull_request17340
2017-04-23 07:12:39serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2017-04-23 05:58:11serhiy.storchakasetmessages: + msg292154
2017-04-23 05:50:24serhiy.storchakasetmessages: + msg292153
2017-04-23 05:50:16serhiy.storchakasetmessages: + msg292152
2017-04-23 05:49:59serhiy.storchakasetpull_requests: + pull_request1374
2017-04-23 05:43:24serhiy.storchakasetpull_requests: + pull_request1373
2017-04-23 05:42:51serhiy.storchakasetpull_requests: + pull_request1372
2017-04-23 05:38:00serhiy.storchakasetmessages: + msg292151
2017-04-22 16:28:26serhiy.storchakasetmessages: + msg292125
2017-04-22 16:09:17terry.reedysetnosy: + serhiy.storchaka

messages: + msg292123
versions: + Python 3.7
2017-04-22 15:08:09serhiy.storchakasetpull_requests: + pull_request1369
2016-01-01 23:47:02martin.pantersetnosy: + martin.panter
messages: + msg257318
2016-01-01 23:01:16terry.reedysetversions: + Python 3.5, Python 3.6, - Python 3.2, Python 3.3
2016-01-01 22:31:30Winterflowersetnosy: + Winterflower
messages: + msg257312
2012-08-18 13:38:03Arfreversetnosy: + Arfrever
2012-08-17 20:18:44terry.reedysetversions: + Python 3.2, Python 3.3
nosy: + terry.reedy, benjamin.peterson, georg.brandl, vstinner

messages: + msg168473

stage: needs patch
2012-08-17 11:18:15Rostyslav.Dzinkosettype: behavior
2012-08-17 11:17:58Rostyslav.Dzinkocreate