classification
Title: Document auto __ne__ generation; provide a use case for non-trivial __ne__
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.6, Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: Arfrever, antocuni, berker.peksag, chris.jerdonek, cvrebert, docs@python, eric.araujo, franck, mark.dickinson, martin.panter, medwards, ncoghlan, python-dev, rbcollins, rhettinger, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2008-11-23 18:21 by terry.reedy, last changed 2015-08-06 22:43 by rbcollins. This issue is now closed.

Files
File name Uploaded Description Edit
default-ne-reflected-priority.patch martin.panter, 2015-01-11 06:27 review
default-ne-reflected-priority.v2.patch martin.panter, 2015-01-25 22:15 review
default-ne-reflected-priority.v3.patch martin.panter, 2015-07-20 04:43 Resolve conflict review
default-ne-reflected-priority.v4.patch martin.panter, 2015-07-20 07:27 review
Messages (20)
msg76270 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2008-11-23 18:21
3.0c3 doc (Basic customization) says
"There are no implied relationships among the comparison operators. The
truth of x==y does not imply that x!=y is false. Accordingly, when
defining __eq__(), one should also define __ne__() so that the operators
will behave as expected. "

In http://mail.python.org/pipermail/python-ideas/2008-October/002235.html
Guido says
"I should also note that part of George's proposal has already been
implemented: if you define __eq__, you get a complementary __ne__ for
free. However it doesn't work the other way around (defining __ne__
doesn't give you __eq__ for free), and there is no similar
relationship for the ordering operators."

And indeed, as Arnaud Delobelle posted on python-list
class A:
    def __init__(self, x):
        self.x = x
    def __eq__(self, other):
        return self.x == other.x

a, b, c = A(1), A(1), A(2)
print(a==b, b==c, c==a) # True, False, False
print(a!=b, b!=c, c!=a) # False, True, True

Suggested revision:
"There is one implied relationship among comparison operators: defining
__eq__ gives an automatic __ne__ (but not the other way).  There is no
similar relationship for the order comparisons.
msg76374 - (view) Author: Michael K. Edwards (medwards) Date: 2008-11-25 00:49
It would be really useful to explain, right in this section, why __ne__
is worth having.  Something along these lines (based on the logic from
Python 2.x -- modify as necessary):

<doctext>

The values most commonly returned by the rich comparison methods are
True, False, and NotImplemented (which tells the Python interpreter to
try a different comparison strategy).  However, it is quite legal and
often useful to return some other value, usually one which can be
coerced to True/False by bool().

For instance, if equality testing of instances of some class is
computationally expensive, that class's implementation of __eq__ may
return a "comparison object" whose __nonzero__ method calculates and
caches the actual boolean value.  Subsequent references to this same
comparison object may be returned for subsequent, logically equivalent
comparisons; the expensive comparison takes place only once, when the
object is first used in a boolean context.  This class's implementation
of __ne__ could return, not just "not (self == other)", but an object
whose __nonzero__ method returns "not (self == other)" -- potentially
delaying the expensive operation until its result is really tested for
boolean truth.

Python allows the programmer to define __ne__ separately from __eq__ for
this and similar reasons.  It is up to the programmer to ensure that
bool(self != other) == (not bool(self == other)), if this is a desired
property.  (One can even imagine situations in which it is appropriate
for neither (self == other) nor (self != other) to be true.  For
instance, a mathematical theorem prover might contain values a, b, c,
... that are formally unknown, and raise an exception when a==b is used
in a boolean context, but allow comparison of M = (a==b) against N =
(a!=b).)

</doctext>

Now that I write this, I see a use for magic __logical_or__,
__logical_and__, and __logical_not__ methods, so that one can postpone
or even avoid the evaluation of expensive/indeterminate comparisons. 
Consider the expression:
  ((a==b) and (c==d)) and ((a!=b) and (d==f))
If my class is designed such that a==b and a!=b cannot both be true,
then I can conclude that this expression is false without evaluating any
of the equality/inequality tests.

Is it too late to request these for Python 3.0?
msg89532 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2009-06-19 23:06
The current paragraph
"There are no implied relationships among the comparison operators. The
truth of x==y does not imply that x!=y is false. Accordingly, when
defining __eq__(), one should also define __ne__() so that the operators
will behave as expected. "
is false.

Please, let us replace it now, for 3.1 release, with the correct
"There is one implied relationship among comparison operators: defining
__eq__ gives an automatic __ne__ (but not the other way).  There is no
similar relationship for the order comparisons."
without waiting for a more extensive rewrite.
msg89533 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-06-19 23:17
One other thought:  The __ne__ method follows automatically from __eq__
only if __ne__ isn't already defined in a superclass.  So, if you're
inheriting from a builtin, it's best to override both.
msg89553 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2009-06-21 01:29
The situation appears to be at least slightly different from what Guido
stated. In 3.x, all classes subclass object, which has .__ne__, so if
that stopped inferred != behavior, it would never happen.

>>> class A:
	def __eq__(s,p): return 1

>>> id(object.__ne__)
10703216
>>> id(A.__ne__)
10703216

No new A.__ne__ added.  But

>>> c,d=object(),object()
>>> c==d
False
>>> c!=d
True
>>> a,b = A(),A()
>>> a==b
1
>>> a!=b
False

So it seems that a!=b *is* evaluated as not a==b rather than as
a.__ne__(b). If so, my revised suggested replacement would be:

"There is one implied relationship among comparison operators: defining
__eq__ causes '!=' to be evaluated as 'not ==' (but not the other way).
 There is no similar relationship for the order comparisons."

I am a bit puzzled though. In
ttp://svn.python.org/view/python/branches/py3k/Python/ceval.c?revision=73066&view=markup
I traced compare_op to cmp_outcome to (in object.c) PyOjbect_RichCompare
to do_richcompare to class specific tp_richcompare and I do not see the
special casing of eq. However, I am newbie at codebase.
msg89591 - (view) Author: Michael K. Edwards (medwards) Date: 2009-06-22 03:47
The implementation you are looking for is in object_richcompare, in
http://svn.python.org/projects/python/branches/py3k/Objects/typeobject.c
.  It would be most accurate to say something like:

    The "object" base class, from which all user-defined classes
inherit, provides a single "rich comparison" method to which all of the
comparison operators (__eq__, __ne__, __lt__, __le__, __ge__, __gt__)
map.  This method returns a non-trivial value (i. e., something other
than NotImplemented) in only two cases:
  * When called as __eq__, if the two objects are identical, this method
returns True.  (If they are not identical, it returns NotImplemented so
that the other object's implementation of __eq__ gets a chance to return
True.)
  * When called as __ne__, it calls the equivalent of "self == other";
if this returns a non-trivial value X, then it returns !X (which is
always either True or False).
msg89592 - (view) Author: Michael K. Edwards (medwards) Date: 2009-06-22 05:08
It would also be useful to point out that there is a shortcut in the
interpreter itself (PyObject_RichCompareBool, in object.c) which checks
the equivalent of id(a) == id(b) and bypasses __eq__/__ne__ if so. 
Since not every call to __eq__ passes through this function, it's fairly
important that implementations of __eq__ return either True or
NotImplemented when id(a) == id(b).  Ditto for extension modules;
anything that installs its own tp_richcompare should handle object
identity and __ne__ in substantially the same way, so that subclass
authors can rely on the documented behavior when overriding __eq__.
msg181630 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-02-07 16:28
Issue #17151 closed as a duplicate of this one.
msg233835 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-11 06:27
Here is a patch that documents the default object.__ne__() implementation. It also documents the subclass priority rules for the reflected comparison methods, which is raised in Issue 22052.

I have made some more tests to verify the relationships exists from __ne__ to __eq__, but no other relationships exist for the other methods. I will include it in my patch in Issue 21408 to avoid the patches conflicting with each other.
msg234601 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-01-24 11:02
While Martin's patch doesn't cover all the vagaries of comparison operations discussed above, it fixes the outright error, and provides an appropriate cross-reference to functools.total_ordering.
msg234605 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-24 12:40
The reference to @functools.total_ordering was actually already there; I just moved it into the paragraph about relationships between the operators. I should also point out that my description of the default __ne__() assumes that Issue 21408 is resolved; the current behaviour is slightly different.

If you think something else could be added to the patch, I’m happy to try and add it. Perhaps the default object.__eq__() behaviour?
msg234696 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-25 22:15
Adding a new patch that just fixes the typo error in the first patch
msg234780 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-26 21:35
See also issue23326.
msg236012 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-02-15 00:12
Issue 21408 has been committed to 3.4 and 3.5 branches, so my patch can now be considered to document the newly fixed behaviour.
msg246955 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-20 04:43
Nick seemed to approve of this, so perhaps it is ready to commit? The new patch just resolves a minor conflict with the current code.
msg246959 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-07-20 05:22
Added comments on Rietveld.
msg246968 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-07-20 07:27
This updated patch adds the clarification about NotImplemented.
msg246969 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-07-20 08:12
LGTM.
msg248155 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-08-06 22:34
New changeset f5069e6e4229 by Robert Collins in branch '3.4':
Issue #4395: Better testing and documentation of binary operators.
https://hg.python.org/cpython/rev/f5069e6e4229

New changeset b9a0165a3de8 by Robert Collins in branch '3.5':
Issue #4395: Better testing and documentation of binary operators.
https://hg.python.org/cpython/rev/b9a0165a3de8

New changeset e56893df8e76 by Robert Collins in branch 'default':
Issue #4395: Better testing and documentation of binary operators.
https://hg.python.org/cpython/rev/e56893df8e76
msg248156 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-06 22:43
Thanks for the patch; applied to 3.4 and up.
History
Date User Action Args
2015-08-06 22:43:02rbcollinssetstatus: open -> closed

nosy: + rbcollins
messages: + msg248156

resolution: fixed
stage: commit review -> resolved
2015-08-06 22:34:53python-devsetnosy: + python-dev
messages: + msg248155
2015-07-20 08:12:44serhiy.storchakasetmessages: + msg246969
2015-07-20 07:27:29martin.pantersetfiles: + default-ne-reflected-priority.v4.patch

messages: + msg246968
2015-07-20 05:22:58serhiy.storchakasetmessages: + msg246959
2015-07-20 04:43:18martin.pantersetfiles: + default-ne-reflected-priority.v3.patch

stage: patch review -> commit review
messages: + msg246955
versions: + Python 3.6
2015-03-19 11:24:37Arfreversetnosy: + Arfrever
2015-02-15 00:12:09martin.pantersetmessages: + msg236012
2015-01-26 21:35:37serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg234780
2015-01-25 22:15:47martin.pantersetfiles: + default-ne-reflected-priority.v2.patch

messages: + msg234696
2015-01-24 12:40:46martin.pantersetmessages: + msg234605
2015-01-24 11:02:00ncoghlansetmessages: + msg234601
2015-01-11 16:17:03berker.peksagsetnosy: + berker.peksag
2015-01-11 06:39:54asvetlovsetstage: needs patch -> patch review
2015-01-11 06:39:37asvetlovsetversions: + Python 3.4, Python 3.5, - Python 3.2, Python 3.3
2015-01-11 06:27:32martin.pantersetfiles: + default-ne-reflected-priority.patch

nosy: + martin.panter
messages: + msg233835

keywords: + patch
2013-02-07 16:28:53mark.dickinsonsetpriority: low -> normal
2013-02-07 16:28:32mark.dickinsonsetnosy: + mark.dickinson, franck
messages: + msg181630
2013-02-07 16:27:09mark.dickinsonlinkissue17151 superseder
2012-12-08 11:01:54chris.jerdoneksetnosy: + chris.jerdonek
2012-12-08 10:39:52ncoghlansetnosy: + antocuni
2012-12-08 10:39:08ncoghlansetnosy: + ncoghlan
2011-11-18 14:32:50eric.araujosetnosy: + eric.araujo
2011-11-15 20:07:02ezio.melottisetversions: + Python 3.3, - Python 3.1
2011-01-12 00:21:00eric.araujosetnosy: + docs@python, - georg.brandl
stage: needs patch
type: behavior
versions: + Python 3.2
2010-09-02 00:45:05rhettingersetpriority: normal -> low
assignee: rhettinger -> terry.reedy
2009-08-16 11:53:25cvrebertsetnosy: + cvrebert
2009-06-22 05:08:35medwardssetmessages: + msg89592
2009-06-22 03:48:00medwardssetmessages: + msg89591
2009-06-21 04:36:52rhettingersetassignee: georg.brandl -> rhettinger
2009-06-21 01:29:51terry.reedysetmessages: + msg89553
2009-06-19 23:17:50rhettingersetnosy: + rhettinger
messages: + msg89533
2009-06-19 23:06:12terry.reedysetmessages: + msg89532
versions: + Python 3.1, - Python 3.0
2008-11-25 00:49:33medwardssetnosy: + medwards
messages: + msg76374
title: Document auto __ne__ generation -> Document auto __ne__ generation; provide a use case for non-trivial __ne__
2008-11-23 18:21:33terry.reedycreate