classification
Title: Comparison operators called in reverse order for subclasses with no override.
Type: Stage: resolved
Components: Documentation Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, mark.dickinson, martin.panter, r.david.murray, serhiy.storchaka
Priority: normal Keywords:

Created on 2014-07-23 21:03 by mark.dickinson, last changed 2020-05-31 13:19 by serhiy.storchaka. This issue is now closed.

Messages (10)
msg223778 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-07-23 21:03
As reported in a StackOverflow question [1]: the order in which the special comparison methods are called seems to be contradictory to the docs [2].  In the following snippet, __eq__ is called with reversed operands first:

>>> class A:
...     def __eq__(self, other):
...         print(type(self), type(other))
...         return True
... 
>>> class B(A):
...     pass
... 
>>> A() == B()
<class '__main__.B'> <class '__main__.A'>
True

However, the docs note that:

"""If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations."""

... which suggests that this reversal should only happen when the subclass B *overrides* A's definition of __eq__ (and indeed that's the usual behaviour for arithmetic operations like __add__).

Looking more closely, that statement in the docs is in the 'numeric-types' section, so it's not clear that its rules should apply to the comparison operators.  But either way, some doc clarification could be useful.


[1] http://stackoverflow.com/q/24919375/270986
[2] https://docs.python.org/3.5/reference/datamodel.html#emulating-numeric-types
msg223789 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-23 22:22
"the subclass provides" doesn't actually imply anything about overriding, I think.  For __eq__, though, that means that *every* class "provides" it.  Indeed, I've always thought of the rule as "the subclass goes first" with no qualification, but that's only true for __eq__.

It looks like the "subclass goes first" note is missing from the __eq__ section.

But see issue 21408.  I find it hard to reason about this algorithm, so I could be completely wrong :)
msg223812 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-07-24 07:19
> "the subclass provides" doesn't actually imply anything about overriding, I think.

Yes, that was the thrust of one of the SO answers.  Unfortunately, that explanation doesn't work for arithmetic operators, though: there an explicit override is necessary.  Here's another example, partly to get away from the extra complication of __eq__ being its own inverse.  After:

    class A(object):
        def __lt__(self, other): return True
        def __gt__(self, other): return False
        def __add__(self, other): return 1729
        def __radd__(self, other): return 42

    class B(A): pass

we get:

    >>> A() + B()
    1729
    >>> A() < B()
    False

So the addition is calling the usual __add__ method first (the special exception in the docs doesn't apply: while B *is* a subclass of A, it doesn't *override* A's __radd__ method).  But the comparison is (surprisingly) calling the __gt__ method first.

So we've got two different rules being followed: one for arithmetic operators, and a different one for comparisons.

This isn't a big deal behaviour-wise: I'm certainly not advocating a behaviour change here.  But it would be nice to document it.
msg223838 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-07-24 14:08
Ah yes.  I remember there being a discussion somewhere about the differences between comparison operator inverses and the arithmetic 'r' methods, but I can't find it at the moment.  I *thought* there was a full discussion of the logic involved in these cases, but I can't find that either.  We need one somewhere that we can crosslink to if it doesn't already exist.
msg233836 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-11 06:36
I have included some rules about the priority for calling reflected operator methods in my patch to Issue 4395
msg248163 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-08-07 00:57
My patch was committed for Python 3.4+. The priority of the comparator methods is now documented at the end of <https://docs.python.org/dev/reference/datamodel.html#richcmpfuncs>. Perhaps all that is left to do here is to apply similar changes to the Python 2 documentation.
msg251397 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-09-23 02:40
Does anyone know enough about Python 2 to propose a fix? I don’t know enough about object classes versus “instance” classes, and potential interference of the __cmp__() method. In Python 2 the order seems to depend on the class type:

(<__main__.A instance at 0x7f730d37f5f0>, <__main__.B instance at 0x7f730d37f518>)
(<__main__.B object at 0x7f730d37dc10>, <__main__.A object at 0x7f730d37d110>)

Or perhaps we should just close this now and forget about Python 2 ;)
msg251410 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2015-09-23 07:02
For Python 2, I think the most we should do is document the behaviour somewhere; changing it in a bugfix release seems both unnecessary and potentially risky.
msg251411 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2015-09-23 07:04
> the most we should do is document the behaviour somewhere

And indeed, perhaps this issue counts as sufficient documentation...
msg370440 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-05-31 13:19
Python 2.7 is no longer supported.
History
Date User Action Args
2020-05-31 13:19:14serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg370440

resolution: fixed
stage: needs patch -> resolved
2015-09-23 07:04:05mark.dickinsonsetmessages: + msg251411
2015-09-23 07:02:07mark.dickinsonsetmessages: + msg251410
2015-09-23 02:40:03martin.pantersetstage: needs patch
messages: + msg251397
versions: - Python 3.4, Python 3.5
2015-08-07 00:57:22martin.pantersetmessages: + msg248163
2015-01-11 06:36:23martin.pantersetnosy: + martin.panter
messages: + msg233836
2014-07-24 14:08:43r.david.murraysetmessages: + msg223838
2014-07-24 07:19:35mark.dickinsonsetmessages: + msg223812
2014-07-23 22:22:08r.david.murraysetnosy: + r.david.murray
messages: + msg223789
2014-07-23 21:03:30mark.dickinsoncreate