This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Tuple comparisons with NaNs are broken
Type: Stage:
Components: Interpreter Core Versions: Python 3.4
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Electro, akira, inducer, mark.dickinson, rhettinger
Priority: normal Keywords:

Created on 2014-06-26 08:23 by Electro, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (11)
msg221600 - (view) Author: Mak Nazečić-Andrlon (Electro) Date: 2014-06-26 08:23
While searching for a way to work around the breakage of the Schwartzian transform in Python 3 (and the resulting awkwardness if you wish to use heapq or bisect, which do not yet have a key argument), I thought of the good old IEEE-754 NaN. Unfortunately, that shouldn't work since lexicographical comparisons shouldn't stop for something comparing False all the time. Nevertheless:

>>> (1, float("nan"), A()) < (1, float("nan"), A())
False
>>> (0, float("nan"), A()) < (1, float("nan"), A())
True

Instead of as in
>>> nan = float("nan")
>>> (1, nan, A()) < (1, nan, A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: A() < A()

(As a side note, PyPy3 does not have this bug.)
msg221603 - (view) Author: Akira Li (akira) * Date: 2014-06-26 14:42
Is the issue that:

  >>> (1, float('nan')) == (1, float('nan'))
  False

but

  >>> nan = float('nan')
  >>> (1, nan) == (1, nan)
  True

?

`nan != nan` therefore it might be expected that `(a, nan) != (a, nan)` [1]:

> The values float('NaN') and Decimal('NaN') are special. The are identical to themselves, x is x but are not equal to themselves, x != x. 

> Tuples and lists are compared lexicographically using comparison of corresponding elements. This means that to compare equal, each element must compare equal and the two sequences must be of the same type and have the same length.
> If not equal, the sequences are ordered the same as their first differing elements.

[1]: https://docs.python.org/3.4/reference/expressions.html#comparisons
msg221604 - (view) Author: Akira Li (akira) * Date: 2014-06-26 14:46
btw, pypy3 (986752d005bb) is broken:

  >>>> (1, float('nan')) == (1, float('nan'))
  True
msg221608 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-06-26 15:23
Python containers are allowed to let identity-imply-equality (the reflesive property of equality).  Dicts, lists, tuples, deques, sets, and frozensets all work this way.  So for your purposes,  you need to use distinct NaN values rather than reusing a single instance of a NaN.
msg221609 - (view) Author: Mak Nazečić-Andrlon (Electro) Date: 2014-06-26 15:34
The bug is that the comparison should throw a TypeError, but does not (for incomparable A).
msg221612 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-06-26 16:02
Python core containers support the invariant:

    assert all(x in c for x in c)

See also:  http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/
msg221663 - (view) Author: Akira Li (akira) * Date: 2014-06-26 23:23
> Python containers are allowed to let identity-imply-equality (the reflesive property of equality).

Is it documented somewhere?

> Dicts, lists, tuples, deques, sets, and frozensets all work this way. 

Is it CPython specific behaviour?
msg221670 - (view) Author: Mak Nazečić-Andrlon (Electro) Date: 2014-06-27 01:05
It's not about equality.

    >>> class A: pass
    ... 
    >>> (float("nan"), A()) < (float("nan"), A())
    False

That < comparison should throw a TypeError, since NaN < NaN is False, in the same way that 0 < 0 is False here:

>>> (0, A()) < (0, A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: A() < A()
msg221685 - (view) Author: Akira Li (akira) * Date: 2014-06-27 12:40
It is about equality. `float('nan') != float('nan')` unlike `0 == 0`.

From msg221603: 

> If not equal, the sequences are ordered the same as their first differing elements.

The result of the expression: `(a, whatever) < (b, whatever)` is defined by 
`a < b` if a and b differs i.e., it is not necessary to compare other elements (though Python language reference doesn't forbid further comparisons. It doesn't specify explicitly the short-circuit behavior for sequence comparisons unlike for `and`, `or` operators that guarantee the lazy (only as much as necessary) evaluation).
msg221704 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-06-27 18:39
FWIW, the logic for tuple ordering is a bit weird due to rich comparisons.   Each pair of elements is first checked for equality (__eq__).  Only if the equality comparison returns False does it call the relevant ordering operations (such as __lt__).   The docs get it right, "If not equal, the sequences are ordered the same as their first differing elements."

In short tuple ordering is different from scalar ordering because it always makes equality tests:
 
   a < b              calls           a.__lt__(b)

in contrast:

   (a, b) < (c, d)    is more like:   if a != c:  return a < c ...
msg221781 - (view) Author: Akira Li (akira) * Date: 2014-06-28 12:22
> (a, b) < (c, d) is more like: if a != c: return a < c ...

except CPython behaves (undocumented?) as:

  b < d if a is c or a == c else a < c

the difference is in the presence of `is` operator (identity
comparison instead of `__eq__`). `nan is nan` therefore `b < d` is
called and raises TypeError for `(nan, A()) < (nan, A())` expression
where `a = c = nan`, `b = A()`, and `d = A()`.

But `(float("nan"), A()) < (float("nan"), A())` is False (no
TypeError) because `a is not c` in this case and `a < c` is called
instead where `a = float('nan')`, `b = A()`, `c = float('nan')`, and
`d = A()`. Plus `(a, b) < (c, d)` evaluation is lazy (undocumented?)
i.e., once `a < c` determines the final result `b < d` is not called.
History
Date User Action Args
2022-04-11 14:58:05adminsetgithub: 66072
2022-03-24 19:25:03inducersetnosy: + inducer
2014-06-28 12:22:33akirasetmessages: + msg221781
2014-06-27 18:39:17rhettingersetmessages: + msg221704
2014-06-27 12:40:04akirasetmessages: + msg221685
2014-06-27 01:05:14Electrosetmessages: + msg221670
2014-06-26 23:23:01akirasetmessages: + msg221663
2014-06-26 16:02:30rhettingersetassignee: rhettinger
messages: + msg221612
2014-06-26 15:34:43Electrosetmessages: + msg221609
2014-06-26 15:23:52rhettingersetstatus: open -> closed

nosy: + rhettinger
messages: + msg221608

resolution: not a bug
2014-06-26 14:46:36akirasetmessages: + msg221604
2014-06-26 14:42:04akirasetnosy: + akira
messages: + msg221603
2014-06-26 11:57:28r.david.murraysetnosy: + mark.dickinson
2014-06-26 08:23:29Electrocreate