Rationale:

"""
IEEE 754 assigns values to all relational expressions involving NaN.
In the syntax of C, the predicate x != y is True but all others, x <
y , x <= y , x == y , x >= y and x > y, are False whenever x or y or
both are NaN, and then all but x != y and x == y are INVALID
operations too and must so signal.
"""
-- Lecture Notes on the Status of IEEE Standard 754 for Binary
Floating-Point Arithmetic by Prof. W. Kahan
http://www.cs.berkeley.edu/~wkahan/ieee754status/ieee754.ps

The problem with faithfully implementing IEEE 754 in Python is that
exceptions in IEEE standard don't have the same meaning as in Python.
 IEEE 754 requires that a value is computed even when the operation
signals an exception.  The program can then decide whether to
terminate computation or propagate the value. In Python, we have to
choose between raising an exception and returning the value.  We
cannot have both.  It appears that in most cases IEEE 754 "INVALID"
exception is treated as a terminating exception by Python and
operations that signal INVALID in IEEE 754 raise an exception in
Python.  Therefore making <, >, etc. raise on NaN while keeping the
status quo for != and == would bring Python floats closer to
compliance with IEEE 754.

See http://mail.python.org/pipermail/python-ideas/2011-April/010057.html for discussion.

An instructive part of the patch is

--- a/Lib/test/test_math.py
+++ b/Lib/test/test_math.py
@@ -174,10 +174,22 @@
                    flags
                   )
 
+def is_negative_zero(x):
+    return x == 0 and math.copysign(1, x) < 0
+
+def almost_equal(value, expected):
+    if math.isfinite(expected) and math.isfinite(value):
+        return abs(value-expected) <= eps
+    if math.isnan(expected):
+        return math.isnan(value)
+    if is_negative_zero(expected):
+        return is_negative_zero(value)
+    return value == expected
+
 class MathTests(unittest.TestCase):
 
     def ftest(self, name, value, expected):
-        if abs(value-expected) > eps:
+        if not almost_equal(value, expected):


Although it may look like proposed change makes it harder to compare floats for approximate equality, the change actually helped to highlight a programming mistake: old ftest() accepts 0.0 where -0.0 is expected.

This is a typical situation when someone attempts to write clever code relying on unusual properties of NaNs. In most cases clever code does not account for all possibilities and it is always hard reason about such code.