Title: Doc for comparison of sequences with non-orderable elements
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Dubslow, docs@python, r.david.murray, rhettinger, terry.reedy
Priority: normal Keywords: patch

Created on 2017-11-23 04:27 by Dubslow, last changed 2019-08-24 17:53 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4514 closed python-dev, 2017-11-23 04:28
PR 15450 merged rhettinger, 2019-08-24 06:27
PR 15466 merged miss-islington, 2019-08-24 17:44
Messages (8)
msg306780 - (view) Author: (Dubslow) Date: 2017-11-23 04:27
In sequence comparisons, the enforcement of reflexivity of elements means that only non-identical elements are actually compared. The docs then note, with example, that non-reflexive elements thus always "compare" equal inside the sequence.
    This patch adds a second corollary, that non-orderable singletons (e.g. None) will also not break sequence comparison.
    Yes, the consequence is logically derivable from the statement "element identity is compared first, and element comparison is performed only for distinct elements", but the first example is given because "For non-reflexive elements, the result is different than for strict element comparison, and may be surprising", which also holds for the example I add here: different from strict element comparison, which may lead to otherwise surprising results (it sure was surprising to me when I expected a list with Nones to fail to compare, hence why I went trawling through the docs). In the manner of the first example, explicit is better than implicit, and (I believe) it will be helpful for readers to have this second consequence demonstrated.
msg306782 - (view) Author: (Dubslow) Date: 2017-11-23 04:35
The PR includes an unrelated one word grammar fix in the same file, that can be removed (by me or by someone else, IDC).

This is possibly backportable but I wouldn't know, and leave such decisions for someone who do.
msg306827 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-11-23 16:12
The surprising thing is the behavior of NaN, which is *not equal* to itself.

The statement about orderability says "...are ordered the same as their first unequal elements".  This is explicit and unambiguous, there is no difference in this context between the number 1 and the singleton None, or the reflexivity enforced on NaN: all are equal to the corresponding element from the other sequence.  The whole point of the paragraph is that *no order test is done until the first unequal element is encountered*.

If we want to make this *more* explicit, I would suggest simply adding the following sentence after the first example in the original paragraph: "This means that reflexive elements that are otherwise unorderable (such as None and NaN) do not trigger a TypeError during a comparison."
msg306928 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-11-24 22:48
This whole section has become a mess an is now more complex that the underlying code.   Adding more caveats, special cases, and atypical examples will make it worse (rather like the U.S. tax code, another example of bad technical writing).

I recommend the whole section be rewritten, extracting the most general rules and with examples that cover the general rules.

There can then be brief separate paragraphs for language lawyers that cover what makes NaNs and None unusual (none of the comparison logic special cases these value -- their interesting behaviors are intrinsic to the object itself).
msg306976 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-11-25 22:49
(Raymond, I wrote this before reading your message, but I believe it provides what you requested.  There is nothing special about None that is relevant to sequence comparison.  Nans just happen to be the only built-in non-reflexive objects (that I know of, thank goodness).)

This issue is about order comparisons *also* being defined on a single object in a sequence and therefore also possibly giving a different result.  Title revised to be clearer and short enough to all be visible.  Demonstration:

>>> a = object()
>>> a == a, a != a  # but a < a, a <= a, a >= a, a > a raise TypeError
(True, False)
>>> la = [a]
>>> la == la, la != la, la < la, la <= la, la >= la, la > la
(True, False, False, True, True, False)

Comparison of two sequences infers from identity all 6 rich comparison results on a single object in corresponding positions of the two sequences.

This has nothing to do with the object being a singleton, other than a singleton being a single object.  The enforcement of reflexivity and enforcement of consistent order on single objects are related, but the latter did not have to follow from the first.  The presentation of order, in the patch, by reference to singletons and reflexivity, does not work.  As I said in review, the example is too long and wrongly placed.  I think using a generic object instead of None would also be better.

I would like to rewrite the sequence comparison paragraphs after the first as something like the following.

Sequences compare lexicographically using comparison of corresponding elements.  The two objects are first compared for identity.  If they are distinct, they are compared normally.  If they are the same object, the self comparisons are inferred: '==', '>=', and '<=' are true, while '!=', '>', and '<' are false.

By design, the inferred self comparisons for sequences sometimes give different results than would strict element comparison.  Instances of an unordered class become ordered with respect to themselves instead of raising a TypeError.

>>> a = object()
>>> a < a          # By default, objects are not ordered
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>=' not supported between instances of 'object' and 'object'
>>> [a] < [a]
False              # Sequence comparison orders an object versus itself

Even anti-reflexive not-a-number values, for example, are treated as reflexive (meaning a == a always).

<insert current nan example> 

This re-write describes sequence comparison in one paragraph, instead of putting the details in the middle of the next paragraph.  The next paragraph describes the two possible consequences of inferring comparisons from identify: a result instead of a raise, and a different result.
msg307107 - (view) Author: (Dubslow) Date: 2017-11-28 04:08
I like Terry's suggestion much better. I've closed my GitHub PR in favor of Terry's change.

My only suggested tweak might be throwing in something like "(e.g." None)", perhaps e.g. "Instances of an unordered class (e.g. None) become ordered...", but even as is, is a major improvement over mine. Thanks.
msg350383 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-24 17:43
New changeset edd21129dd304e178ca8be82ba689488dfb58276 by Raymond Hettinger in branch 'master':
bpo-32118:  Simplify docs for sequence comparison (GH-15450)
msg350385 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-24 17:53
New changeset 0ad85681de639792751ea53ec964d87d4ad45d71 by Raymond Hettinger (Miss Islington (bot)) in branch '3.8':
bpo-32118:  Simplify docs for sequence comparison (GH-15450) (#15466)
Date User Action Args
2019-08-24 17:53:33rhettingersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-08-24 17:53:11rhettingersetmessages: + msg350385
2019-08-24 17:44:05miss-islingtonsetpull_requests: + pull_request15158
2019-08-24 17:43:57rhettingersetmessages: + msg350383
2019-08-24 06:27:42rhettingersetpull_requests: + pull_request15141
2017-11-28 04:08:36Dubslowsetmessages: + msg307107
2017-11-27 23:22:38rhettingersetassignee: docs@python -> rhettinger
2017-11-25 22:49:18terry.reedysetnosy: + terry.reedy

messages: + msg306976
title: Docs: add note about sequence comparisons containing non-orderable elements -> Doc for comparison of sequences with non-orderable elements
2017-11-24 22:48:20rhettingersetnosy: + rhettinger
messages: + msg306928
2017-11-23 16:12:57r.david.murraysetnosy: + r.david.murray
messages: + msg306827
2017-11-23 04:35:42Dubslowsettype: enhancement
messages: + msg306782
2017-11-23 04:28:15python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request4451
2017-11-23 04:27:01Dubslowcreate