classification
Title: datetime.datetime operator methods are not subclass-friendly
Type: behavior Stage:
Components: Extension Modules Versions: Python 3.4
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: amaury.forgeotdarc, belopolsky, mark.dickinson, stingray
Priority: normal Keywords: patch

Created on 2008-03-10 19:09 by stingray, last changed 2013-01-12 11:38 by belopolsky. This issue is now closed.

Files
File name Uploaded Description Edit
datetime.diff belopolsky, 2008-03-10 22:05 patch against revision 61343 review
Messages (14)
msg63446 - (view) Author: Paul Komkoff (stingray) Date: 2008-03-10 19:09
The datetime.datetime class overrides some arithmetic operations for it
to be able to add or subtract timedeltas. However, the result of A + B
operation, where A is instance of a subclass of datetime and B is
timedelta instance will be always the instance of base datetime.

This is extremely annoying and requires to override arithmetic operators
and writing a lots of rubbish to replace the datetime base object with
type(self)
msg63452 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-03-10 22:05
This is trivial to implement (see attached) and IMHO is a good idea.

The next question, however is whether similar changes should be made to 
timedelta arithmetics.  Timedelta case is not so clearcut because of the 
usual dilemma of what the type of a+b should be when a and b are 
instances of two different subclasses of timedelta.
msg63453 - (view) Author: Paul Komkoff (stingray) Date: 2008-03-10 22:21
I just checked the astimezone method - it also does this.

As with timedelta... well, it's not critical for me now but it worth
thinking about :)
msg63454 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-03-10 23:05
Recently, the similar issue1562 "Decimal can't be subclassed useful" was
rejected. In the discussion I found a reference to a former post, which
precisely deals with datetime and timedelta:
http://mail.python.org/pipermail/python-list/2005-January/300791.html

The main argument is that the """base class has no idea what
requirements may exist for invoking a subclass's constructor"""

All python types behave this way: int, float, lists.
msg63456 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-03-11 00:29
Invoking a subclass's constructor is only an issue when subclass adds 
data members.  In this case, arithmetic methods need to be overridden.  
Note that my patch does not make __add__ and friends invoke subclass' 
constructor, only subclass' tp_alloc.

Existing code already does this in some cases.  For example,

>>> class d(datetime): pass
... 
>>> d.strptime('20080310', '%Y%m%d')
d(2008, 3, 10, 0, 0)
>>> d.now()
d(2008, 3, 10, 20, 27, 6, 303147)

I think date/datetime present a particularly compelling case for 
departing from the general rule.  These classes are minimal by design 
and need to be extended in many applications.
msg107410 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-09 18:51
I would like to take another shot at this. The link in Amaury's closing comment no longer works, so here is the relevant post:

"""
Returning same type as self for arithmetic in subclasses

Tim Peters tim.peters at gmail.com 
Sat Jan 8 02:09:27 CET 2005

[Max M]
> """
> I subclass datetime and timedelta
> 
> >>> dt = myDatetime(1970,1,1)
> >>> type(dt)
> <class 'dtime.myDatetime'>
> 
> >>> td = myTimedelta(hours=1)
> >>> type(td)
> <class 'dtime.myTimedelta'>
> 
> But when I do arithmetic with these classes, they return datetime and
> timedelta,
...
> >>> new_time = dt + td
> >>> new_time
> datetime.datetime(1970, 1, 1, 1, 0)
> 
> >>> type(new_time)
> <type 'datetime.datetime'>

Yes, and all builtin Python types work that way.  For example,
int.__add__ or float.__add__ applied to a subclass of int or float
will return an int or float; similarly for a subclass of str.  This
was Guido's decision, based on that an implementation of any method in
a base class has no idea what requirements may exist for invoking a
subclass's constructor.  For example, a subclass may restrict the
values of constructor arguments, or require more arguments than a base
class constructor; it may permute the order of positional arguments in
the base class constructor; it may even be "a feature" that a subclass
constructor gives a different meaning to an argument it shares with
the base class constructor.  Since there isn't a way to guess, Python
does a safe thing instead.

> where I want them to return myDatetime and myTimedelta
>
> So I wondered if there was a simlpler way to coerce the result into my
> desired types rather than overwriting the __add__, __sub__ etc. methods?

Generally speaking, no.  But I'm sure someone will torture you with a
framework that purports to make it easy <wink>.
"""  http://mail.python.org/pipermail/python-list/2005-January/925838.html


As I explained in my previous post, the same argument, "base class has no idea what requirements may exist for invoking a subclass's constructor", applies to class methods, but they nevertheless consistently construct subclass instances:

>>> class d(datetime): pass
>>> d.utcfromtimestamp(0)
d(1970, 1, 1, 0, 0)
>>> d.fromtimestamp(0)
d(1969, 12, 31, 19, 0)
>>> d.combine(date(1,1,1), time(1,1))
d(1, 1, 1, 1, 1)


Similar example for the date class:

>>> class Date(date): pass
>>> Date.fromordinal(1)
Date(1, 1, 1)


In my view it is hard to justify that for a Date instance d, and integer days, Date.fromordinal(d.toordinal() + days) happily produces a Date instance, but d + timedelta(days) returns a basic date instance.
msg107414 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-06-09 20:31
If you want to challenge Guido's design decision, I think python-dev would be the place to do it.
msg107416 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-09 21:03
> If you want to challenge Guido's design decision, I think
> python-dev would be the place to do it.

Do you have a link supporting that it was "Guido's design decision"?  This decision must have been made around class/type unification, but I don't remember reading about it in Guido's essays.

I don't want to make a fool of myself by coming to python-dev with this unprepared. :-)
msg107417 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-09 21:05
I also believe something needs to be fixed here in any case.  Either Date.fromordinal(..) should return date or Date(..) + timedelta(..) should return Date.
msg107418 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-06-09 21:08
> Do you have a link [...]

Nope.  Just going on Tim's description of it as "Guido's decision".  I've no idea of the history, and I don't particularly recall any recent relevant python-dev discussions.
msg108060 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2010-06-17 20:30
There is a difference between methods and overridden operators (slightly) in terms of cognitive understanding. I mean creating a new instance from a timestamp seems like an operation on the object by the object. Addition, though, seems like a creation of a new object by the two objects working together, which suggests contravariance.

Best I can think of. Otherwise ask on python-dev since Guido called the operator overriding expectation.
msg108061 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-06-17 20:52
I had understood that the rule was that alternate constructors should be classmethods, for consistency with __new__.  (Well, except that __new__ is actually a staticmethod, of course... )

E.g., after "class MyDecimal(Decimal): pass", MyDecimal('2.3') produces a MyDecimal instance, and by analogy MyDecimal.from_float(2.3) should also produce a MyDecimal instance.  It's exactly the same type of function as the class constructor.

I don't think it would do any harm to get clarification from python-dev on the underlying reasons.
msg125979 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-11 01:51
Note that before r82065, python prototype, now available as Lib/datetime.py had binary operations implemented to return subclass instances.  Here is annotated pre-r82065 code:


 39876 gvanrossum     def __add__(self, other):
 39876 gvanrossum         if isinstance(other, timedelta):
 39928 gvanrossum             return self.__class__(self.__days + other.__days,
 39876 gvanrossum                                   self.__seconds + other.__seconds,
 39876 gvanrossum                                   self.__microseconds + other.__microseconds)
 40207    tim_one         return NotImplemented
 39876 gvanrossum
msg179783 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-01-12 10:15
Alexander: can this be closed as "wont fix"?
History
Date User Action Args
2013-01-12 11:38:19belopolskysetstatus: open -> closed
resolution: wont fix
2013-01-12 10:15:33mark.dickinsonsetmessages: + msg179783
2013-01-11 16:20:51brett.cannonsetnosy: - brett.cannon

versions: + Python 3.4, - Python 3.3
2011-01-11 01:51:24belopolskysetnosy: brett.cannon, amaury.forgeotdarc, mark.dickinson, belopolsky, stingray
messages: + msg125979
versions: + Python 3.3, - Python 3.2
2010-06-17 20:52:17mark.dickinsonsetmessages: + msg108061
2010-06-17 20:30:54brett.cannonsetnosy: + brett.cannon
messages: + msg108060
2010-06-09 21:08:35mark.dickinsonsetmessages: + msg107418
2010-06-09 21:05:59belopolskysetmessages: + msg107417
2010-06-09 21:03:17belopolskysetmessages: + msg107416
2010-06-09 20:31:13mark.dickinsonsetmessages: + msg107414
2010-06-09 18:51:06belopolskysetstatus: closed -> open

assignee: belopolsky
components: + Extension Modules, - Library (Lib)
versions: + Python 3.2, - Python 2.5
nosy: + mark.dickinson

messages: + msg107410
resolution: wont fix -> (no value)
2008-03-11 00:29:24belopolskysetmessages: + msg63456
2008-03-10 23:05:50amaury.forgeotdarcsetstatus: open -> closed
resolution: wont fix
messages: + msg63454
nosy: + amaury.forgeotdarc
2008-03-10 22:21:58stingraysetmessages: + msg63453
2008-03-10 22:05:22belopolskysetfiles: + datetime.diff
keywords: + patch
messages: + msg63452
nosy: + belopolsky
2008-03-10 19:09:50stingraycreate