Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document auto __ne__ generation; provide a use case for non-trivial __ne__ #48645

Closed
terryjreedy opened this issue Nov 23, 2008 · 20 comments
Closed
Assignees
Labels
docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error

Comments

@terryjreedy
Copy link
Member

BPO 4395
Nosy @rhettinger, @terryjreedy, @mdickinson, @ncoghlan, @rbtcollins, @merwok, @cjerdonek, @berkerpeksag, @vadmium, @serhiy-storchaka, @antocuni
Files
  • default-ne-reflected-priority.patch
  • default-ne-reflected-priority.v2.patch
  • default-ne-reflected-priority.v3.patch: Resolve conflict
  • default-ne-reflected-priority.v4.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2015-08-06.22:43:02.223>
    created_at = <Date 2008-11-23.18:21:33.030>
    labels = ['type-bug', 'docs']
    title = 'Document auto __ne__ generation; provide a use case for non-trivial __ne__'
    updated_at = <Date 2015-08-06.22:43:02.215>
    user = 'https://github.com/terryjreedy'

    bugs.python.org fields:

    activity = <Date 2015-08-06.22:43:02.215>
    actor = 'rbcollins'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2015-08-06.22:43:02.223>
    closer = 'rbcollins'
    components = ['Documentation']
    creation = <Date 2008-11-23.18:21:33.030>
    creator = 'terry.reedy'
    dependencies = []
    files = ['37671', '37859', '39954', '39958']
    hgrepos = []
    issue_num = 4395
    keywords = ['patch']
    message_count = 20.0
    messages = ['76270', '76374', '89532', '89533', '89553', '89591', '89592', '181630', '233835', '234601', '234605', '234696', '234780', '236012', '246955', '246959', '246968', '246969', '248155', '248156']
    nosy_count = 17.0
    nosy_names = ['rhettinger', 'terry.reedy', 'mark.dickinson', 'ncoghlan', 'rbcollins', 'eric.araujo', 'Arfrever', 'medwards', 'cvrebert', 'chris.jerdonek', 'docs@python', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'franck', 'antocuni']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue4395'
    versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

    @terryjreedy
    Copy link
    Member Author

    3.0c3 doc (Basic customization) says
    "There are no implied relationships among the comparison operators. The
    truth of x==y does not imply that x!=y is false. Accordingly, when
    defining __eq__(), one should also define __ne__() so that the operators
    will behave as expected. "

    In http://mail.python.org/pipermail/python-ideas/2008-October/002235.html
    Guido says
    "I should also note that part of George's proposal has already been
    implemented: if you define __eq__, you get a complementary __ne__ for
    free. However it doesn't work the other way around (defining __ne__
    doesn't give you __eq__ for free), and there is no similar
    relationship for the ordering operators."

    And indeed, as Arnaud Delobelle posted on python-list
    class A:
    def __init__(self, x):
    self.x = x
    def __eq__(self, other):
    return self.x == other.x

    a, b, c = A(1), A(1), A(2)
    print(a==b, b==c, c==a) # True, False, False
    print(a!=b, b!=c, c!=a) # False, True, True

    Suggested revision:
    "There is one implied relationship among comparison operators: defining
    __eq__ gives an automatic __ne__ (but not the other way). There is no
    similar relationship for the order comparisons.

    @terryjreedy terryjreedy added the docs Documentation in the Doc dir label Nov 23, 2008
    @medwards
    Copy link
    Mannequin

    medwards mannequin commented Nov 25, 2008

    It would be really useful to explain, right in this section, why __ne__
    is worth having. Something along these lines (based on the logic from
    Python 2.x -- modify as necessary):

    <doctext>

    The values most commonly returned by the rich comparison methods are
    True, False, and NotImplemented (which tells the Python interpreter to
    try a different comparison strategy). However, it is quite legal and
    often useful to return some other value, usually one which can be
    coerced to True/False by bool().

    For instance, if equality testing of instances of some class is
    computationally expensive, that class's implementation of __eq__ may
    return a "comparison object" whose __nonzero__ method calculates and
    caches the actual boolean value. Subsequent references to this same
    comparison object may be returned for subsequent, logically equivalent
    comparisons; the expensive comparison takes place only once, when the
    object is first used in a boolean context. This class's implementation
    of __ne__ could return, not just "not (self == other)", but an object
    whose __nonzero__ method returns "not (self == other)" -- potentially
    delaying the expensive operation until its result is really tested for
    boolean truth.

    Python allows the programmer to define __ne__ separately from __eq__ for
    this and similar reasons. It is up to the programmer to ensure that
    bool(self != other) == (not bool(self == other)), if this is a desired
    property. (One can even imagine situations in which it is appropriate
    for neither (self == other) nor (self != other) to be true. For
    instance, a mathematical theorem prover might contain values a, b, c,
    ... that are formally unknown, and raise an exception when a==b is used
    in a boolean context, but allow comparison of M = (a==b) against N =
    (a!=b).)

    </doctext>

    Now that I write this, I see a use for magic __logical_or__,
    __logical_and__, and __logical_not__ methods, so that one can postpone
    or even avoid the evaluation of expensive/indeterminate comparisons.
    Consider the expression:
    ((a==b) and (c==d)) and ((a!=b) and (d==f))
    If my class is designed such that a==b and a!=b cannot both be true,
    then I can conclude that this expression is false without evaluating any
    of the equality/inequality tests.

    Is it too late to request these for Python 3.0?

    @medwards medwards mannequin changed the title Document auto __ne__ generation Document auto __ne__ generation; provide a use case for non-trivial __ne__ Nov 25, 2008
    @terryjreedy
    Copy link
    Member Author

    The current paragraph
    "There are no implied relationships among the comparison operators. The
    truth of x==y does not imply that x!=y is false. Accordingly, when
    defining __eq__(), one should also define __ne__() so that the operators
    will behave as expected. "
    is false.

    Please, let us replace it now, for 3.1 release, with the correct
    "There is one implied relationship among comparison operators: defining
    __eq__ gives an automatic __ne__ (but not the other way). There is no
    similar relationship for the order comparisons."
    without waiting for a more extensive rewrite.

    @rhettinger
    Copy link
    Contributor

    One other thought: The __ne__ method follows automatically from __eq__
    only if __ne__ isn't already defined in a superclass. So, if you're
    inheriting from a builtin, it's best to override both.

    @terryjreedy
    Copy link
    Member Author

    The situation appears to be at least slightly different from what Guido
    stated. In 3.x, all classes subclass object, which has .__ne__, so if
    that stopped inferred != behavior, it would never happen.

    >>> class A:
    	def __eq__(s,p): return 1
    
    >>> id(object.__ne__)
    10703216
    >>> id(A.__ne__)
    10703216

    No new A.__ne__ added. But

    >>> c,d=object(),object()
    >>> c==d
    False
    >>> c!=d
    True
    >>> a,b = A(),A()
    >>> a==b
    1
    >>> a!=b
    False

    So it seems that a!=b *is* evaluated as not a==b rather than as
    a.__ne__(b). If so, my revised suggested replacement would be:

    "There is one implied relationship among comparison operators: defining
    __eq__ causes '!=' to be evaluated as 'not ==' (but not the other way).
    There is no similar relationship for the order comparisons."

    I am a bit puzzled though. In
    ttp://svn.python.org/view/python/branches/py3k/Python/ceval.c?revision=73066&view=markup
    I traced compare_op to cmp_outcome to (in object.c) PyOjbect_RichCompare
    to do_richcompare to class specific tp_richcompare and I do not see the
    special casing of eq. However, I am newbie at codebase.

    @rhettinger rhettinger assigned rhettinger and unassigned birkenfeld Jun 21, 2009
    @medwards
    Copy link
    Mannequin

    medwards mannequin commented Jun 22, 2009

    The implementation you are looking for is in object_richcompare, in
    http://svn.python.org/projects/python/branches/py3k/Objects/typeobject.c
    . It would be most accurate to say something like:

    The "object" base class, from which all user-defined classes
    

    inherit, provides a single "rich comparison" method to which all of the
    comparison operators (eq, __ne__, __lt__, __le__, __ge__, __gt__)
    map. This method returns a non-trivial value (i. e., something other
    than NotImplemented) in only two cases:

    • When called as __eq__, if the two objects are identical, this method
      returns True. (If they are not identical, it returns NotImplemented so
      that the other object's implementation of __eq__ gets a chance to return
      True.)
    • When called as __ne__, it calls the equivalent of "self == other";
      if this returns a non-trivial value X, then it returns !X (which is
      always either True or False).

    @medwards
    Copy link
    Mannequin

    medwards mannequin commented Jun 22, 2009

    It would also be useful to point out that there is a shortcut in the
    interpreter itself (PyObject_RichCompareBool, in object.c) which checks
    the equivalent of id(a) == id(b) and bypasses __eq__/ne if so.
    Since not every call to __eq__ passes through this function, it's fairly
    important that implementations of __eq__ return either True or
    NotImplemented when id(a) == id(b). Ditto for extension modules;
    anything that installs its own tp_richcompare should handle object
    identity and __ne__ in substantially the same way, so that subclass
    authors can rely on the documented behavior when overriding __eq__.

    @rhettinger rhettinger assigned terryjreedy and unassigned rhettinger Sep 2, 2010
    @merwok merwok added the type-bug An unexpected behavior, bug, or error label Jan 12, 2011
    @mdickinson
    Copy link
    Member

    Issue bpo-17151 closed as a duplicate of this one.

    @vadmium
    Copy link
    Member

    vadmium commented Jan 11, 2015

    Here is a patch that documents the default object.__ne__() implementation. It also documents the subclass priority rules for the reflected comparison methods, which is raised in bpo-22052.

    I have made some more tests to verify the relationships exists from __ne__ to __eq__, but no other relationships exist for the other methods. I will include it in my patch in bpo-21408 to avoid the patches conflicting with each other.

    @ncoghlan
    Copy link
    Contributor

    While Martin's patch doesn't cover all the vagaries of comparison operations discussed above, it fixes the outright error, and provides an appropriate cross-reference to functools.total_ordering.

    @vadmium
    Copy link
    Member

    vadmium commented Jan 24, 2015

    The reference to @functools.total_ordering was actually already there; I just moved it into the paragraph about relationships between the operators. I should also point out that my description of the default __ne__() assumes that bpo-21408 is resolved; the current behaviour is slightly different.

    If you think something else could be added to the patch, I’m happy to try and add it. Perhaps the default object.__eq__() behaviour?

    @vadmium
    Copy link
    Member

    vadmium commented Jan 25, 2015

    Adding a new patch that just fixes the typo error in the first patch

    @serhiy-storchaka
    Copy link
    Member

    See also bpo-23326.

    @vadmium
    Copy link
    Member

    vadmium commented Feb 15, 2015

    bpo-21408 has been committed to 3.4 and 3.5 branches, so my patch can now be considered to document the newly fixed behaviour.

    @vadmium
    Copy link
    Member

    vadmium commented Jul 20, 2015

    Nick seemed to approve of this, so perhaps it is ready to commit? The new patch just resolves a minor conflict with the current code.

    @serhiy-storchaka
    Copy link
    Member

    Added comments on Rietveld.

    @vadmium
    Copy link
    Member

    vadmium commented Jul 20, 2015

    This updated patch adds the clarification about NotImplemented.

    @serhiy-storchaka
    Copy link
    Member

    LGTM.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Aug 6, 2015

    New changeset f5069e6e4229 by Robert Collins in branch '3.4':
    Issue bpo-4395: Better testing and documentation of binary operators.
    https://hg.python.org/cpython/rev/f5069e6e4229

    New changeset b9a0165a3de8 by Robert Collins in branch '3.5':
    Issue bpo-4395: Better testing and documentation of binary operators.
    https://hg.python.org/cpython/rev/b9a0165a3de8

    New changeset e56893df8e76 by Robert Collins in branch 'default':
    Issue bpo-4395: Better testing and documentation of binary operators.
    https://hg.python.org/cpython/rev/e56893df8e76

    @rbtcollins
    Copy link
    Member

    Thanks for the patch; applied to 3.4 and up.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    docs Documentation in the Doc dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    9 participants