classification
Title: equality not symmetric for subclasses of datetime.date and datetime.datetime
Type: behavior Stage: needs patch
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution: postponed
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: belopolsky, eric.snow, jackdied, jess.austin, ncoghlan, p-ganssle, ysj.ray
Priority: low Keywords:

Created on 2009-03-19 00:51 by jess.austin, last changed 2018-07-05 15:58 by p-ganssle.

Files
File name Uploaded Description Edit
issue5516.diff jess.austin, 2009-03-23 20:20 patch against trunk for datetimemodule.c and test_datetime.py
issue5516_trunk.diff jackdied, 2009-03-26 16:51 udiff against 2.7 trunk
Messages (19)
msg83798 - (view) Author: Jess Austin (jess.austin) Date: 2009-03-19 00:51
While the datetime.date and datetime.datetime classes consistently
handle mixed-type comparison, their subclasses do not:

>>> from datetime import date, datetime, time
>>> d = date.today()
>>> dt = datetime.combine(d, time())
>>> d == dt
False
>>> dt == d
False
>>> class D(date):
...     pass
... 
>>> class DT(datetime):
...     pass
... 
>>> d = D.today()
>>> dt = DT.combine(d, time())
>>> d == dt
True
>>> dt == d
False

I think this is due to the premature "optimization" of using memcmp() in
date_richcompare().
msg84028 - (view) Author: Jess Austin (jess.austin) Date: 2009-03-23 20:20
The attached patch fixes this issue, and updates the tests.  Contrary to
my initial impression, it seems that a previous developer knew of this
behavior and thought it correct; see the comment of the test I deleted.
 I left memcmp() in.
msg84184 - (view) Author: Jack Diederich (jackdied) * (Python committer) Date: 2009-03-26 16:51
+1

Patch and tests work for me.  Uploaded a patch that is identical except
the file paths are fixed.

Was the old behavior stable across compilers anyway?  It memcmpared two
different structs and IIRC only the first item of each struct is
guaranteed to be at the start of the memory location.  No?

With this patch only same-struct objects are memcmpared.
msg103854 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2010-04-21 15:27
There is another inconsistency that this patch does not seem to cure.  With patch applied and D and DT defined as in OP,

>>> D(1900,1,1) > DT(1900,1,1)
True

but

>>> DT(1900,1,1) < D(1900,1,1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare DT to D

and

>>> date(1900,1,1) < datetime(1900,1,1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare datetime.datetime to datetime.date

Note that without the patch,

>>> D(1900,1,1) > DT(1900,1,1)
False

but both behaviors seem to be wrong.
msg103864 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2010-04-21 16:51
Upon further reflection I am -1 on this patch.  First, as implemented this patch changes behavior of an explicit invocation of date.__eq__.  The patch proposes to remove the following tests:

-        # Neverthelss, comparison should work with the base-class (date)
-        # projection if use of a date method is forced.
-        self.assert_(as_date.__eq__(as_datetime))
-        different_day = (as_date.day + 1) % 20 + 1
-        self.assert_(not as_date.__eq__(as_datetime.replace(day=
-                                                     different_day)))

Second, the patch introduces dependence of the baseclass method (date_richcompare) on a particular subclass (PyDateTime_Check).  This is against OOP principles.

I am not sure how the "equality not symmetric" issue can be fixed.  In my opinion current datetime implementation is fighting OOP and violating the substitution principle in an attempt to prevent date to datetime comparison.  I would prefer seeing one of two things: either datetime not inheriting from date or making datetime to date comparison compare date part alone.  Once you stop fighting OOP principles, symmetry of equality for subclasses will follow from that for the base class automatically.

Given that either of these solutions means a major change, I think it is best to leave the things as they are now.
msg103881 - (view) Author: Jess Austin (jess.austin) Date: 2010-04-21 18:33
To be systematic, without the patch:

>>> D(1900, 1, 1) > DT(1900, 1, 1)
False
>>> D(1900, 1, 1) < DT(1900, 1, 1)
False
>>> DT(1900, 1, 1) > D(1900, 1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare DT to D
>>> DT(1900, 1, 1) < D(1900, 1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare DT to D


with the patch:

>>> D(1900, 1, 1) > DT(1900, 1, 1)
True
>>> D(1900, 1, 1) < DT(1900, 1, 1)
False
>>> DT(1900, 1, 1) > D(1900, 1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare DT to D
>>> DT(1900, 1, 1) < D(1900, 1, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't compare DT to D


It might seem like the latter behavior is marginally better, but really this is just a mess, since a date-datetime comparison TypeErrors in all directions.  I appreciate Alexander's more experienced perspective, but it's not obvious to me that this problem is insoluble simply due to OOP algebra.  I'm going to keep tinkering with this to see if there isn't a way to satisfy his concerns AND fix these bugs WITHOUT breaking the established (and admittedly anti-OOP) behavior that dates are not equal to datetimes.

(Incidentally, the test I removed still seems to be an arbitrary ad-hoc requirement that subclasses of date behave differently than date itself.  I don't see how one could rely on this given the other inconsistencies with date subclasses, and so violating this in order to fix more serious problems seems acceptable.)

I'm reminded of the set and frozenset situation, which seems almost dual to this one.  set and frozenset don't inherit from each other, but they do compare.  This seems to bite you only when you try to redefine comparison in subclasses.
msg103886 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2010-04-21 18:46
On Wed, Apr 21, 2010 at 2:33 PM, Jess Austin <report@bugs.python.org> wrote:
..
> It might seem like the latter behavior is marginally better, but really this is just a mess, since a date-datetime comparison TypeErrors in all
> directions.  I appreciate Alexander's more experienced perspective, but it's not obvious to me that this problem is insoluble simply due to OOP
> algebra.  I'm going to keep tinkering with this to see if there isn't a way to satisfy his concerns AND fix these bugs WITHOUT breaking the
> established (and admittedly anti-OOP) behavior that dates are not equal to datetimes.
>

I certainly don't have a proof that this is impossible, so best of
luck.  Note, however that the problematic behavior is due to D/DT
classes implementor's choice not to derive DT from D.  Whether
resulting violation of the symmetry of equality is a bug in python or
D/DT implementation is at least an open question.
msg104433 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-04-28 14:39
I think I'll concur with the "this is a mess" assessment.

Given that state of affairs, punting on this until 3.2 (at the earliest).
msg106497 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-05-26 00:47
I am leaning towards "won't fix".  Any objections?
msg106501 - (view) Author: Jess Austin (jess.austin) Date: 2010-05-26 01:47
Could you provide some reasoning for such a resolution?  I had thought that "won't fix" indicated that the issue wasn't actually an error in behavior.

I grant that most people will never see this particular error, but it is an error.
msg106503 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-05-26 02:19
> Could you provide some reasoning for such a resolution? 
> I had thought that "won't fix" indicated that the issue
> wasn't actually an error in behavior.

No, that would be "invalid."  IMO, "won't fix" is for bugs were cost of fixing them outweighs the benefits.  Here is a typical example: issue8309 "Sin(x) is Wrong".

Here, however I am torn between "won't fix" and "invalid."  As I said in my previous comment:

"""
Note, however that the problematic behavior is due to D/DT
classes implementor's choice not to derive DT from D.  Whether
resulting violation of the symmetry of equality is a bug in python or
D/DT implementation is at least an open question.
"""

I don't mind keeping this open if there is a hope that someone will come up with a working solution.  The current patch is not a solution.
msg106505 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-05-26 03:27
I'd suggest leaving it open - the current situation is definitely suboptimal, but it is likely to take some close scrutiny to get it to behave nicely.
msg106507 - (view) Author: ysj.ray (ysj.ray) Date: 2010-05-26 04:26
Date and Datetime comparison is not defined and not documented, and until we find a nice way to implement the comparison, we should just let this comparison raise NotImpelemented.
msg106508 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-05-26 04:56
Deprecating the feature for 3.x is certainly an option. May be a little drastic though.
msg108603 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-25 15:10
> Deprecating the feature for 3.x is certainly an option.
> May be a little drastic though.

How drastic would be to stop subclassing datetime from date in 3.2?  After all, we don't subclass float form int.
msg108684 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-06-26 03:20
If you can articulate the benefits of splitting them apart, it would be worth making the case for doing so on python-dev. I'm having a hard time picturing what anyone could be doing such that it would break their code, but it's still definitely a backwards incompatible change.
msg195030 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-08-12 22:11
This bit me today (under 2.7).
msg195034 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2013-08-12 22:51
Eric,

Could you share details of your use-case?  My experience with subclassing from basic python types including date/time has been mostly negative.  The problem is that when I subclass, I want to inherit the rich set of operations such as +, -, *, etc., and add a few methods of my own.  After that, I want to always use instances of my subclass instead of the stdlib one.  This does not work because adding instances of my subclass returns an instance of the superclass unless I override __add__ explicitly. (See   #2267.)  This kills all benefits of subclassing as compared to containment.

These days I try to stay away from subclassing date/time, int/float, or anything like that and thus have little incentive to resolve this issue.  And it does not look like we have a workable solution.
msg195035 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-08-12 23:06
I'm doing some string-based serialization of datetimes and need to be able to specify the type somewhat declaratively.  So I'm using a datetime subclass.  This is more or less the code I'm using:


class Timestamp(datetime.datetime):

    def __new__(cls, raw_value, *args, **kwargs):
        if not args and not kwargs:
            return cls.fromtimestamp(int(raw_value))
        else:
            return super(Timestamp, cls).__new__(cls, raw_value,
                                                 *args, **kwargs)

    def __str__(self):
        return str(int(time.mktime(self.timetuple())))


Incidently, the whole equality testing thing didn't actually cause a problem.  It was comparing against the result of `datetime.utcnow()` which has microseconds (and my Timestamp instance didn't).  Clearing out the microseconds resolved the failure so I wasn't actually bitten by this issue after all.
History
Date User Action Args
2018-07-05 15:58:15p-gansslesetnosy: + p-ganssle
2016-09-18 10:08:35mark.dickinsonsetnosy: - mark.dickinson
2016-09-10 18:35:30belopolskysetversions: + Python 3.7, - Python 3.3
2013-08-12 23:06:05eric.snowsetmessages: + msg195035
2013-08-12 22:51:54belopolskysetmessages: + msg195034
2013-08-12 22:11:54eric.snowsetnosy: + eric.snow
messages: + msg195030
2011-01-12 01:59:51belopolskysetkeywords: - patch
nosy: mark.dickinson, ncoghlan, belopolsky, jackdied, jess.austin, ysj.ray
2010-12-01 18:35:59belopolskysetresolution: postponed
versions: + Python 3.3, - Python 3.2
2010-06-26 03:20:08ncoghlansetmessages: + msg108684
2010-06-25 15:10:28belopolskysetmessages: + msg108603
2010-05-26 04:56:54ncoghlansetmessages: + msg106508
2010-05-26 04:26:15ysj.raysetnosy: + ysj.ray
messages: + msg106507
2010-05-26 03:27:20ncoghlansetmessages: + msg106505
2010-05-26 02:19:06belopolskysetpriority: normal -> low

messages: + msg106503
stage: needs patch
2010-05-26 01:47:44jess.austinsetmessages: + msg106501
2010-05-26 00:47:01belopolskysetassignee: belopolsky
messages: + msg106497
nosy: + belopolsky, - Alexander.Belopolsky
versions: - Python 2.6, Python 2.5
2010-04-28 14:39:23ncoghlansetassignee: ncoghlan -> (no value)
messages: + msg104433
versions: + Python 3.2, - Python 2.4, Python 3.0, Python 3.1, Python 2.7
2010-04-21 18:46:24Alexander.Belopolskysetmessages: + msg103886
2010-04-21 18:33:49jess.austinsetmessages: + msg103881
2010-04-21 16:53:54pitrousetnosy: + mark.dickinson
2010-04-21 16:51:27Alexander.Belopolskysetmessages: + msg103864
2010-04-21 15:27:35Alexander.Belopolskysetnosy: + Alexander.Belopolsky
messages: + msg103854
2010-01-13 10:10:57ncoghlansetassignee: ncoghlan

nosy: + ncoghlan
2009-03-26 16:51:10jackdiedsetfiles: + issue5516_trunk.diff
nosy: + jackdied
messages: + msg84184

2009-03-23 20:20:51jess.austinsetfiles: + issue5516.diff
keywords: + patch
messages: + msg84028
2009-03-19 00:54:30jess.austinsettitle: equality not reflexive for subclasses of datetime.date and datetime.datetime -> equality not symmetric for subclasses of datetime.date and datetime.datetime
2009-03-19 00:51:42jess.austinsettitle: equality not reflixive for subclasses of datetime.date and datetime.datetime -> equality not reflexive for subclasses of datetime.date and datetime.datetime
2009-03-19 00:51:23jess.austincreate