classification
Title: Avoid raising OverflowError in bool()
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: wont fix
Dependencies: 29839 Superseder:
Assigned To: Nosy List: mark.dickinson, rhettinger, serhiy.storchaka, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2017-03-17 20:27 by serhiy.storchaka, last changed 2017-04-23 07:14 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
bool-overflow.diff serhiy.storchaka, 2017-03-17 20:35
bool-no-overflow-double-call.patch serhiy.storchaka, 2017-04-22 15:18
Pull Requests
URL Status Linked Edit
PR 1211 closed serhiy.storchaka, 2017-04-20 07:29
Messages (8)
msg289781 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-17 20:27
For now bool() raises OverflowError if __bool__ is not defined and __len__ returns large value.

>>> class A:
...     def __len__(self):
...         return 1 << 1000
... 
>>> bool(A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer
>>> bool(range(1<<1000))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t

Proposed patch makes bool() returning True if len() raises OverflowError.

This is an alternate solution of issue28876.
msg290139 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-03-24 20:49
https://docs.python.org/3/library/functions.html#bool refers to
https://docs.python.org/3/library/stdtypes.html#truth

The latter says that values are true ("All other values are considered true") unless one of certain conditions holds.  For user-defined classes, the condition is that the class defines a __bool__() or __len__() method and that the first of those methods returns the bool False or integer zero.

I easily interpret this as meaning that bool(x) (should) *always* return True or False.  In particular, for user classes, any exception in user-coded __bool__ or __len__ (should be) included in "does not return integer zero or bool value False".  This would mean that 'True' would truly be the default return for Bool().

There is currently an unstated exception for raised Exceptions.  This issue proposes an exception to the exception for OverflowErrors (once negative lengths consistently raise ValueErrors and never OverflowErrors).  While this sensible in itself, I am completely happy with the added complication.  I would like to either reconsider the exception for Exceptions or make it explicit.

Patch has new text and What's New entry.  Added logic in object.c looks correct.
msg290606 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-03-27 13:03
Serhiy: Can you please create a pull request? It would be easier to review.
msg290615 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-27 14:06
This issue depends on issue29839. Tests are failed until the patch of issue29839 is merged.
msg292064 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-21 16:34
Hum, I dislike this change since it's non-obvious what/who is raising the OverflowError. If an object calls a function in __len__() and the function raises OverflowError, should we consider that object is "true"? In temptation to guess, I prefer to not guess but passthrough the exception.

If you want to support bool(range(1<<1000)), we need to get the result of __len__() as a Python object rather than a C Py_ssize_t.

Maybe, if __len__() raises an OverflowError: call again the len(), but using the "__len__" method instead of the slot?
msg292121 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-22 15:18
I had similar doubts about this patch and needed opinions of other core developers.

> Maybe, if __len__() raises an OverflowError: call again the len(), but using the "__len__" method instead of the slot?

Following patch implements this idea. I don't like it because it is too complicated.

I think that we should either document that raising an OverflowError by __len__() is normal and interpreted as true in Boolean context, or document that __len__() should return a value not larger than sys.maxsize, otherwise len() and bool() can raise an OverflowError (see issue15718).
msg292140 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-22 22:05
If someone wants to return a value larger than maxsize and support bool():
it is already possible right now by defining a __bool__ method no? If yes,
I suggest to only document this CPython implementation detail (consequence
of slots, for efficiency) and suggest to use __bool__ for such corner case.

I also dislike retrying to call "__len__" method instead of the slot. It
can have annoying side effects.
msg292156 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-23 07:14
This was documented in issue15718.
History
Date User Action Args
2017-04-23 07:14:38serhiy.storchakasetstatus: open -> closed
resolution: wont fix
messages: + msg292156

stage: patch review -> resolved
2017-04-22 22:05:05vstinnersetmessages: + msg292140
2017-04-22 15:18:18serhiy.storchakasetfiles: + bool-no-overflow-double-call.patch

messages: + msg292121
2017-04-21 16:34:42vstinnersetmessages: + msg292064
2017-04-20 07:29:15serhiy.storchakasetpull_requests: + pull_request1333
2017-03-27 14:06:27serhiy.storchakasetmessages: + msg290615
2017-03-27 13:03:41vstinnersetnosy: + vstinner
messages: + msg290606
2017-03-24 20:49:53terry.reedysetnosy: + terry.reedy
messages: + msg290139
2017-03-17 20:35:34serhiy.storchakasetfiles: + bool-overflow.diff
keywords: + patch
dependencies: + Avoid raising OverflowError in len() when __len__() returns negative large value
2017-03-17 20:27:26serhiy.storchakacreate