I have a class where I want to override the __eq__
method. It seems to make sense that I should override the __ne__
method as well. Should I implement __ne__
as the negation of __eq__
as such or is it a bad idea?
class A:
def __init__(self, state):
self.state = state
def __eq__(self, other):
return self.state == other.state
def __ne__(self, other):
return not self.__eq__(other)
Python, should I implement
__ne__()
operator based on__eq__
?
==
, not __eq__
In Python 3, !=
is the negation of ==
by default, so you are not even required to write a __ne__
, and the documentation is no longer opinionated on writing one.
Generally speaking, for Python 3-only code, don't write one unless you need to overshadow the parent implementation, e.g. for a builtin object.
That is, keep in mind Raymond Hettinger's comment:
The
__ne__
method follows automatically from__eq__
only if__ne__
isn't already defined in a superclass. So, if you're inheriting from a builtin, it's best to override both.
If you need your code to work in Python 2, follow the recommendation for Python 2 and it will work in Python 3 just fine.
In Python 2, Python itself does not automatically implement any operation in terms of another - therefore, you should define the __ne__
in terms of ==
instead of the __eq__
.
E.G.
class A(object):
def __eq__(self, other):
return self.value == other.value
def __ne__(self, other):
return not self == other # NOT `return not self.__eq__(other)`
See proof that
__ne__()
operator based on __eq__
and __ne__
in Python 2 at allprovides incorrect behavior in the demonstration below.
The documentation for Python 2 says:
There are no implied relationships among the comparison operators. The truth of
x==y
does not imply thatx!=y
is false. Accordingly, when defining__eq__()
, one should also define__ne__()
so that the operators will behave as expected.
So that means that if we define __ne__
in terms of the inverse of __eq__
, we can get consistent behavior.
This section of the documentation has been updated for Python 3:
By default,
__ne__()
delegates to__eq__()
and inverts the result unless it isNotImplemented
.
and in the "what's new" section, we see this behavior has changed:
!=
now returns the opposite of==
, unless==
returnsNotImplemented
.
For implementing __ne__
, we prefer to use the ==
operator instead of using the __eq__
method directly so that if self.__eq__(other)
of a subclass returns NotImplemented
for the type checked, Python will appropriately check other.__eq__(self)
From the documentation:
The
NotImplemented
objectThis type has a single value. There is a single object with this value. This object is accessed through the built-in name
NotImplemented
. Numeric methods and rich comparison methods may return this value if they do not implement the operation for the operands provided. (The interpreter will then try the reflected operation, or some other fallback, depending on the operator.) Its truth value is true.
When given a rich comparison operator, if they're not the same type, Python checks if the other
is a subtype, and if it has that operator defined, it uses the other
's method first (inverse for <
, <=
, >=
and >
). If NotImplemented
is returned, then it uses the opposite's method. (It does not check for the same method twice.) Using the ==
operator allows for this logic to take place.
Semantically, you should implement __ne__
in terms of the check for equality because users of your class will expect the following functions to be equivalent for all instances of A.:
def negation_of_equals(inst1, inst2):
"""always should return same as not_equals(inst1, inst2)"""
return not inst1 == inst2
def not_equals(inst1, inst2):
"""always should return same as negation_of_equals(inst1, inst2)"""
return inst1 != inst2
That is, both of the above functions should always return the same result. But this is dependent on the programmer.
__ne__
based on __eq__
:First the setup:
class BaseEquatable(object):
def __init__(self, x):
self.x = x
def __eq__(self, other):
return isinstance(other, BaseEquatable) and self.x == other.x
class ComparableWrong(BaseEquatable):
def __ne__(self, other):
return not self.__eq__(other)
class ComparableRight(BaseEquatable):
def __ne__(self, other):
return not self == other
class EqMixin(object):
def __eq__(self, other):
"""override Base __eq__ & bounce to other for __eq__, e.g.
if issubclass(type(self), type(other)): # True in this example
"""
return NotImplemented
class ChildComparableWrong(EqMixin, ComparableWrong):
"""__ne__ the wrong way (__eq__ directly)"""
class ChildComparableRight(EqMixin, ComparableRight):
"""__ne__ the right way (uses ==)"""
class ChildComparablePy3(EqMixin, BaseEquatable):
"""No __ne__, only right in Python 3."""
Instantiate non-equivalent instances:
right1, right2 = ComparableRight(1), ChildComparableRight(2)
wrong1, wrong2 = ComparableWrong(1), ChildComparableWrong(2)
right_py3_1, right_py3_2 = BaseEquatable(1), ChildComparablePy3(2)
(Note: while every second assertion of each of the below is equivalent and therefore logically redundant to the one before it, I'm including them to demonstrate that order does not matter when one is a subclass of the other.)
These instances have __ne__
implemented with ==
:
assert not right1 == right2
assert not right2 == right1
assert right1 != right2
assert right2 != right1
These instances, testing under Python 3, also work correctly:
assert not right_py3_1 == right_py3_2
assert not right_py3_2 == right_py3_1
assert right_py3_1 != right_py3_2
assert right_py3_2 != right_py3_1
And recall that these have __ne__
implemented with __eq__
- while this is the expected behavior, the implementation is incorrect:
assert not wrong1 == wrong2 # These are contradicted by the
assert not wrong2 == wrong1 # below unexpected behavior!
Note that this comparison contradicts the comparisons above (not wrong1 == wrong2
).
>>> assert wrong1 != wrong2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
and,
>>> assert wrong2 != wrong1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
__ne__
in Python 2For evidence that you should not skip implementing __ne__
in Python 2, see these equivalent objects:
>>> right_py3_1, right_py3_1child = BaseEquatable(1), ChildComparablePy3(1)
>>> right_py3_1 != right_py3_1child # as evaluated in Python 2!
True
The above result should be False
!
The default CPython implementation for __ne__
is in typeobject.c
in object_richcompare
:
case Py_NE:
/* By default, __ne__() delegates to __eq__() and inverts the result,
unless the latter returns NotImplemented. */
if (Py_TYPE(self)->tp_richcompare == NULL) {
res = Py_NotImplemented;
Py_INCREF(res);
break;
}
res = (*Py_TYPE(self)->tp_richcompare)(self, other, Py_EQ);
if (res != NULL && res != Py_NotImplemented) {
int ok = PyObject_IsTrue(res);
Py_DECREF(res);
if (ok < 0)
res = NULL;
else {
if (ok)
res = Py_False;
else
res = Py_True;
Py_INCREF(res);
}
}
break;
__ne__
uses __eq__
?Python 3's default __ne__
implementation detail at the C level uses __eq__
because the higher level ==
(PyObject_RichCompare) would be less efficient - and therefore it must also handle NotImplemented
.
If __eq__
is correctly implemented, then the negation of ==
is also correct - and it allows us to avoid low level implementation details in our __ne__
.
Using ==
allows us to keep our low level logic in one place, and avoid addressing NotImplemented
in __ne__
.
One might incorrectly assume that ==
may return NotImplemented
.
It actually uses the same logic as the default implementation of __eq__
, which checks for identity (see do_richcompare and our evidence below)
class Foo:
def __ne__(self, other):
return NotImplemented
__eq__ = __ne__
f = Foo()
f2 = Foo()
And the comparisons:
>>> f == f
True
>>> f != f
False
>>> f2 == f
False
>>> f2 != f
True
Don't take my word for it, let's see what's more performant:
class CLevel:
"Use default logic programmed in C"
class HighLevelPython:
def __ne__(self, other):
return not self == other
class LowLevelPython:
def __ne__(self, other):
equal = self.__eq__(other)
if equal is NotImplemented:
return NotImplemented
return not equal
def c_level():
cl = CLevel()
return lambda: cl != cl
def high_level_python():
hlp = HighLevelPython()
return lambda: hlp != hlp
def low_level_python():
llp = LowLevelPython()
return lambda: llp != llp
I think these performance numbers speak for themselves:
>>> import timeit
>>> min(timeit.repeat(c_level()))
0.09377292497083545
>>> min(timeit.repeat(high_level_python()))
0.2654011140111834
>>> min(timeit.repeat(low_level_python()))
0.3378178110579029
This makes sense when you consider that low_level_python
is doing logic in Python that would otherwise be handled on the C level.
Another answerer writes:
Aaron Hall’s implementation
not self == other
of the__ne__
method is incorrect as it can never returnNotImplemented
(not NotImplemented
isFalse
) and therefore the__ne__
method that has priority can never fall back on the__ne__
method that does not have priority.
Having __ne__
never return NotImplemented
does not make it incorrect. Instead, we handle prioritization with NotImplemented
via the check for equality with ==
. Assuming ==
is correctly implemented, we're done.
not self == other
used to be the default Python 3 implementation of the__ne__
method but it was a bug and it was corrected in Python 3.4 on January 2015, as ShadowRanger noticed (see issue #21408).
Well, let's explain this.
As noted earlier, Python 3 by default handles __ne__
by first checking if self.__eq__(other)
returns NotImplemented
(a singleton) - which should be checked for with is
and returned if so, else it should return the inverse. Here is that logic written as a class mixin:
class CStyle__ne__:
"""Mixin that provides __ne__ functionality equivalent to
the builtin functionality
"""
def __ne__(self, other):
equal = self.__eq__(other)
if equal is NotImplemented:
return NotImplemented
return not equal
This is necessary for correctness for C level Python API, and it was introduced in Python 3, making
__ne__
methods in this patch to close Issue 21408 and__ne__
methods in the follow-on cleanup removed hereredundant. All relevant __ne__
methods were removed, including ones implementing their own check as well as ones that delegate to __eq__
directly or via ==
- and ==
was the most common way of doing so.
Our persistent critic provides a pathological example to make the case for handling NotImplemented
in __ne__
, valuing symmetry above all else. Let's steel-man the argument with a clear example:
class B:
"""
this class has no __eq__ implementation, but asserts
any instance is not equal to any other object
"""
def __ne__(self, other):
return True
class A:
"This class asserts instances are equivalent to all other objects"
def __eq__(self, other):
return True
>>> A() == B(), B() == A(), A() != B(), B() != A()
(True, True, False, True)
So, by this logic, in order to maintain symmetry, we need to write the complicated __ne__
, regardless of Python version.
class B:
def __ne__(self, other):
return True
class A:
def __eq__(self, other):
return True
def __ne__(self, other):
result = other.__eq__(self)
if result is NotImplemented:
return NotImplemented
return not result
>>> A() == B(), B() == A(), A() != B(), B() != A()
(True, True, True, True)
Apparently we should give no mind that these instances are both equal and not equal.
I propose that symmetry is less important than the presumption of sensible code and following the advice of the documentation.
However, if A had a sensible implementation of __eq__
, then we could still follow my direction here and we would still have symmetry:
class B:
def __ne__(self, other):
return True
class A:
def __eq__(self, other):
return False # <- this boolean changed...
>>> A() == B(), B() == A(), A() != B(), B() != A()
(False, False, True, True)
For Python 2 compatible code, use ==
to implement __ne__
. It is more:
In Python 3 only, use the low-level negation on the C level - it is even more simple and performant (though the programmer is responsible for determining that it is correct).
Again, do not write low-level logic in high level Python.