This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Clarify conditions under which float(x) with large x raises OverflowError
Type: behavior Stage:
Components: Documentation Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: cykerway, docs@python, eric.smith, mark.dickinson, steven.daprano
Priority: normal Keywords:

Created on 2021-12-24 13:17 by cykerway, last changed 2022-04-11 14:59 by admin.

Messages (9)
msg409143 - (view) Author: Cyker Way (cykerway) * Date: 2021-12-24 13:17
Acccording to:

>    If the argument is outside the range of a Python float, an OverflowError will be raised.

It is well known that the maximum value in IEEE 754 binary64 format is:

    >>> M = ((1<<53)-1)<<(1023-52)

So `(M+1)` will be out of range. But `float(M+1)` gives me:

    >>> float(M+1)

No OverflowError is thrown. Contrast this with:

    >>> float(M+M)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OverflowError: int too large to convert to float

In another text:

>   ...Almost all machines today (November 2000) use IEEE-754 floating point arithmetic, and almost all platforms map Python floats to IEEE-754 “double precision”...

Is Python not following IEEE 754 binary64 format or something missing here?
msg409146 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2021-12-24 14:08
float(x) performs rounding. Only if the value after rounding is out of range is OverflowError raised.

M + 1, when correctly rounded to the nearest even float, gives sys.float_info.max. Likewise for M+2, M+3 etc all the way to M+K where K equals 2**960 before the correctly rounded result of float(M+K) overflows.

Anything less than K=2**960 is too small to be distinguished from M when added to it. So there's no bug here, the behaviour is correct for IEEE-754, although perhaps the documentation is misleading.

Mark, is my explanation right?
msg409148 - (view) Author: Cyker Way (cykerway) * Date: 2021-12-24 16:09
If python's float strictly adheres to IEEE 754 binary64 (is that so?), then the problem may be caused by rounding. However, even in that case I still don't get the benefits of doing range check on a rounded input but not the input itself. Does IEEE 754 itself give any specifications about range check and rounding during such type conversion?
msg409150 - (view) Author: Cyker Way (cykerway) * Date: 2021-12-24 16:33
IEEE 754, 7.4 Overflow:

>   The overflow exception shall be signaled if and only if the destination format’s largest finite number is exceeded in magnitude by what would have been the rounded floating-point result (see 4) were the exponent range unbounded...

Were this the basis of the float implementation, not raising an overflow exception seems to be the correct behavior? Then this is not a bug, but the python float doc might better say "If the *rounded* argument is outside..."
msg409151 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-24 17:11
Yes, exactly: Python's intentionally following the normal IEEE 754 rules for rounding a value to the binary64 format using the round-ties-to-even rounding rule, as formalised in section 7.4 of IEEE 754-2019 (and quoted by @cykerway). These are the exact same rules that are followed for conversion from str to float (where we return `inf` rather than raise `OverflowError` for large values, but the overflow boundary is the same), or conversion from Fraction to float, or conversion from Decimal to float, etc.

> the python float doc might better say "If the *rounded* argument is outside..."

Docs are hard. I think there's a danger that that word "rounded" would cause more confusion than it alleviates - to me, it suggests that there's some kind of rounding going on *before* conversion to float, rather than *as part of* the conversion to float. This isn't a language specification document, so it's not reasonable to give a perfectly accurate description of what happens - the actual meaning would be lost in the mass of details. (In this case, it would also be rather hard to be precise, given that we have to allow for platforms that aren't using IEEE 754.) I'm not seeing an obvious way to improve the docs here.
msg409152 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-24 17:15
If we wanted to make a change, I think the part of the docs that I'd target would be this sentence:

> a floating point number with the same value (within Python’s floating point precision) is returned

It's that "same value (within Python's floating point precision)" bit that I'd consider changing. We could consider replacing it with something along the lines that "an integer argument is rounded to the nearest float", possibly with an additional note that under the assumption of IEEE 754 binary64 format, we follow the usual IEEE 754 rules.
msg409163 - (view) Author: Cyker Way (cykerway) * Date: 2021-12-24 23:40
Alright that helps. I guess I now understand what's happening here. Here are the two numbers in question:

>>> M = int('1'*53+'0'*971, 2)
>>> N = int('1'*53+'0'+'1'*970, 2)

M is the largest number in binary64 range, while N is the largest number that does not emit an OverflowError when converted to binary64. N+1 will emit that error. N-M == 2**970-1 which means N is larger than M that caused the confusion.
msg409172 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-12-25 17:41
Changing to a documentation issue.
msg409200 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-12-26 15:33
I think the documentation reads better as it currently is. Does this really cause any practical confusion?

As Mark notes, we can't specify things exactly here: that would obfuscate all of the things we're actually trying to say.
Date User Action Args
2022-04-11 14:59:53adminsetgithub: 90331
2021-12-26 15:33:52eric.smithsetnosy: + eric.smith
messages: + msg409200
2021-12-25 17:41:25mark.dickinsonsetassignee: docs@python

components: + Documentation, - Interpreter Core
title: float(x) with large x not raise OverflowError -> Clarify conditions under which float(x) with large x raises OverflowError
nosy: + docs@python
versions: + Python 3.11
messages: + msg409172
resolution: not a bug ->
2021-12-24 23:40:21cykerwaysetmessages: + msg409163
2021-12-24 17:16:15mark.dickinsonsetresolution: not a bug
2021-12-24 17:15:57mark.dickinsonsetmessages: + msg409152
2021-12-24 17:11:52mark.dickinsonsetmessages: + msg409151
2021-12-24 16:33:42cykerwaysetmessages: + msg409150
2021-12-24 16:09:55cykerwaysetmessages: + msg409148
2021-12-24 14:08:37steven.dapranosetnosy: + mark.dickinson, steven.daprano
messages: + msg409146
2021-12-24 13:17:25cykerwaycreate