This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: 2**53+1 != float(2**53+1)
Type: behavior Stage: resolved
Components: Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: PedanticHacker, christian.heimes, mark.dickinson, rhettinger, scoder, steven.daprano, tim.peters
Priority: normal Keywords:

Created on 2021-05-06 09:47 by scoder, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (9)
msg393077 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2021-05-06 09:47
I'm not sure if I should consider this a bug, but I'd at least frown at the behaviour, so I thought I'd bring this up here.

Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
>>> 2**53 == float(2**53)
True
>>> float(2**53+1) == float(2**53+1)
True
>>> 2**53+1 == float(2**53+1)
False

This probably has something to do with the 52bit exponent of double precision floats. But the way I would have expected this to work is that a comparison of an integer to a float would first convert the integer to a float, and then compare the two floating point values. That's also what the code says. However, comparing the actual two floating point values gives the expected result, whereas letting the comparison do the conversion internally leads to a different outcome. The code in float_richcompare() uses a vastly more complex implementation than PyLong_AsDouble(), which is likely the reason for this difference in behaviour.

I found this on the system Python on 64bit Ubuntu 20.04, but also tried with a self-built 3.10a7+, giving the same result. I'm only setting the target to 3.10/11 since a potential behavioural change would likely not find its way back to 3.9 and earlier any more.
msg393080 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2021-05-06 10:17
This is not a bug. It is *literally correct* that the int 9007199254740993 is not equal to the float 9007199254740992.0 so I really don't know why you would desire a different result. If you want to compare two floats, compare two floats, not an int and a float.

And it's not a *potential behaviour change", it would be an actual behaviour change, and a serious regression.

Stefan states:

> But the way I would have expected this to work is that a comparison of an integer to a float would first convert the integer to a float, and then compare the two floating point values. That's also what the code says.


No it doesn't. What the code says:

    2**53 + 1 == float(2**53 + 1)

not:

    float(2**53 + 1) == float(2**53 + 1)

There's no conversion on the left hand side. The code literally says to compare an int to a float.


In general, Python is pretty good at giving as close to mathematically correct results as possible. If you want to compare values approximately as floats, you should explicitly convert them to floats.
msg393081 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-05-06 10:32
2**53-1 is also the largest safe rational number in JavaScript and JSON where double precision floats and ints behave the same, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER
msg393086 - (view) Author: Boštjan Mejak (PedanticHacker) * Date: 2021-05-06 12:02
I would compare it like this:

>>> from decimal import Decimal
>>> 2**53 + 1 == Decimal(2**53 + 1)
True
msg393087 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-05-06 12:40
> But the way I would have expected this to work is that a comparison of an integer to a float would first convert the integer to a float

I have a vague memory that that's the way it *did* work once upon a time, many many decades ago. But an equality comparison between int and float that simply converted the int to a float would break transitivity of equality, leading to issues with containment in sets, dicts and lists. Given

n = 2**53
x = 2.**53

if equality comparison worked as you describe you'd have n == x and x == n + 1, so to keep transitivity you'd have to make n == n + 1.

In short, the behaviour is very much deliberate.

> That's also what the code says.

Do you have a pointer? This may be a comment bug.
msg393088 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-05-06 12:49
> I have a vague memory that that's the way it *did* work once upon a time

Here we go: https://bugs.python.org/issue513866
And the corresponding commit, from September 2004: https://github.com/python/cpython/commit/307fa78107c39ffda1eb4ad18201d25650354c4e
msg393089 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-05-06 12:51
Closing here, since this isn't a bug. But I'd still like to understand better what Stefan meant by "That's also what the code says."
msg393093 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2021-05-06 14:13
> I really don't know why you would desire a different result.

I found it surprising that a comparison uses a different method of conversion than the (obvious) user-side conversion, with a different outcome. This seems to be implementation details leaking into the user side.


> "That's also what the code says."

I wasn't referring to a specific comment. What I meant was that the code in float_richcompare() goes to great length trying to convert the integer to a float in a safe way so that it can compare the two values.

https://github.com/python/cpython/blob/985ac016373403e8ad41f8d563c4355ffa8d49ff/Objects/floatobject.c#L403

I now see that it goes the other way at the end, though. If both values have the same order of magnitude, then it actually converts the float to a PyLong instead, thus choosing one of the integer values out of the value range that the float spans and comparing that. That's where the difference originates.


> If you want to compare values approximately as floats, you should explicitly convert them to floats.

As I wrote, "I'm not sure if I should consider this a bug", because it's an area that we could just define as "out of bounds behaviour" and "user, you're on your own".

The net effect is that some integers will never equal a floating point value, even though the floating point value does its very best to represent that integer.

I can live with considering the current behaviour "as good as it gets, because there is no right way to do it".

Thank you for your comments.
msg393104 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2021-05-06 16:10
[Stefan]
> I found it surprising that a comparison uses a different
> method of conversion than the (obvious) user-side
> conversion, with a different outcome. This seems to be
> implementation details leaking into the user side.

It's "spirit of 754", though, so any "principled" implementation would do the same. That is, part of the spirit of 754 is to deliver the infinitely price result _when_ the infinitely precise result is representable.

So, in particular,

> The net effect is that some integers will never equal
> a floating point value, even though the floating point
> value does its very best to represent that integer.

in fact for "almost no" Python ints `i` do i == float(i), because "almost all" unbounded ints `i` lose information when converted to finite-precision float (so with infinite precision they're not equal).
History
Date User Action Args
2022-04-11 14:59:45adminsetgithub: 88220
2021-05-06 16:10:34tim.peterssetmessages: + msg393104
2021-05-06 14:13:36scodersetmessages: + msg393093
2021-05-06 12:51:55mark.dickinsonsetstatus: open -> closed
resolution: not a bug
messages: + msg393089

stage: resolved
2021-05-06 12:49:57mark.dickinsonsetmessages: + msg393088
2021-05-06 12:40:30mark.dickinsonsetmessages: + msg393087
2021-05-06 12:02:00PedanticHackersetnosy: + PedanticHacker
messages: + msg393086
2021-05-06 10:32:16christian.heimessetnosy: + christian.heimes
messages: + msg393081
2021-05-06 10:17:41steven.dapranosetnosy: + steven.daprano
messages: + msg393080
2021-05-06 09:49:30scodersetnosy: + tim.peters
2021-05-06 09:47:53scodercreate