classification
Title: datetime: timedelta(minutes = i) silently fails with numpy.int32 input
Type: behavior Stage: committed/rejected
Components: Library (Lib) Versions: Python 2.7
process
Status: pending Resolution: invalid
Dependencies: Superseder: Py_TPFLAGS_LONG_SUBCLASS is not documented
View: 9307
Assigned To: teoliphant Nosy List: LambertDW, amaury.forgeotdarc, barronh, belopolsky, grubert, teoliphant, terry.reedy
Priority: normal Keywords:

Created on 2009-03-11 15:50 by barronh, last changed 2010-12-01 18:46 by belopolsky.

Messages (13)
msg83470 - (view) Author: Barron Henderson (barronh) Date: 2009-03-11 15:50
Initializing a timedelta object with numpy.int32 arguments give mixed
results; it fails for days keyword, gives bad results for minutes, and
give correct results for seconds/microseconds.  Failure confirmed on
Linux i686 (Py 2.5.2; numpy 1.2.1) and OS X 10.5.6 (Py 2.5.1;
1.2.1).Test case below:

from datetime import timedelta
from numpy import int32
from numpy import int32
from datetime import timedelta

assert timedelta(seconds = 36) == timedelta(seconds = int32(36))
print 'pass 36 sec'
assert timedelta(microseconds = 36) == timedelta(microseconds = int32(36))
print 'pass 36 usec'
assert timedelta(minutes = 35) == timedelta(minutes = int32(35))
print 'pass 35 min'
assert timedelta(minutes = 36) == timedelta(minutes = int32(36))
print 'pass 36 min' # returns bad value
assert timedelta(days = 36) == timedelta(days = int32(36))
print 'pass 36 days' # fails SystemError: Objects/longobject.c:223
msg83471 - (view) Author: David W. Lambert (LambertDW) Date: 2009-03-11 16:06
With older versions of each timedelta rejects the data type.  Maybe
that's a good resolution?

$ /usr/local/bin/python2.4 
Python 2.4.2 (#2, Jul  7 2006, 10:20:47) 
[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.version.version
'0.9.8'
>>> from numpy import int32
>>> from datetime import timedelta
>>> assert timedelta(seconds = 36) == timedelta(seconds = int32(36))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported type for timedelta seconds component: int32scalar
>>>
msg88022 - (view) Author: engelbert gruber (grubert) * Date: 2009-05-18 09:25
testing with 2.6 and numpy 1.3 already ::

   timedelta(seconds = int32(36)) 
   TypeError: unsupported type for timedelta seconds component: 
numpy.int32

But not if I apply a change to numpy to fix issue2263
then ``timedelta(days = int32(36))`` and ``timedelta(minutes = 
int32(36))`` result in wrong timedeltas.
msg109352 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-07-05 20:15
This is a bit strange.  timedelta is accepting int subclass instances:


>>> class I(int):
...    pass
... 
>>> timedelta(days = I(36))
datetime.timedelta(36)


but not numpy.unt_:


>>> timedelta(days = int_(36))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported type for timedelta days component: numpy.int64

which is also a subclass of int:

>>> isinstance(int_(36), int)
True

This is probably a numpy quirk rather than python issue, but is probably worth another look.
msg110788 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-07-19 17:55
As I suspected, datetime not recognizing numpy.int_ as a valid input is a numpy issue.  Unlike regular int subclasses, numpy.int_ does not have Py_TPFLAGS_INT_SUBCLASS flag set:


>>> numpy.int_.__flags__ & (1<<23)
0

>>> class foo(int): pass
... 
>>> foo.__flags__ & (1<<23)
8388608

What python can improve in this area is documentation.  See issue9307.
msg110833 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-19 23:08
Py_TPFLAGS_INT_SUBCLASS is an implementation detail, and extension modules should not have to be aware of it.
Does Numpy correctly call PyType_Ready()?
msg110865 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-07-20 05:08
> Does Numpy correctly call PyType_Ready()?

As far as I can tell, it does:


#define DUAL_INHERIT(child, parent1, parent2)                           \
    Py##child##ArrType_Type.tp_base = &Py##parent2##ArrType_Type;       \
    Py##child##ArrType_Type.tp_bases =                                  \
        Py_BuildValue("(OO)", &Py##parent2##ArrType_Type,               \
                      &Py##parent1##_Type);                             \
    if (PyType_Ready(&Py##child##ArrType_Type) < 0) {                   \
        PyErr_Print();                                                  \
        PyErr_Format(PyExc_SystemError,                                 \
                     "could not initialize Py%sArrType_Type",           \
                     #child);                                           \
        return -1;                                                      \
    }                                                                   \
    Py##child##ArrType_Type.tp_hash = Py##parent1##_Type.tp_hash;
    ...
    DUAL_INHERIT(Int, Int, SignedInteger);

Could it be the multiple inheritance that causes the problem here?
msg110870 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-20 06:27
multiple inheritance should not be a problem: there can be only one "dominant base", which is 'int' in this case.
someone with a debugger should step into this call to PyType_Ready() and see why it does not set the flag correctly (at the end of inherit_special())
msg112912 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-04 22:26
Inconsistent headers ;-)
Should this be closed as invalid and superceded,
or open without resolution?
I cannot tell from discussion; one of you please choose.
msg112913 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-08-04 22:33
This is clearly an invalid issue for python, but I wanted to keep it open until I or someone else gets a chance to report it to numpy project.  I was hoping to close this with a link to numpy tracker.
msg112941 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-05 01:53
A link can be added at any time. Closing does nothing except remove an issue from searches of open issues. Please close this or allow me to. There is a cost to keeping things like this open -- the puzzlement of anyone reading it and the time cost to triage maintainers like me ;-)
msg112992 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-08-05 16:05
Terry,

Realistically, though, if you close this issue it is virtually guaranteed that not further action will be taken.  I have not been involved with numpy for a couple of years now, so it would take me some time to find their tracker and restore my credentials or open a new account.  If someone more familiar with the current numpy development comes across this issue, I would appreciate if he or she would make a report.

If you think keeping this issue open in this state is too much burden on triagers, feel free to close it.  IMO, however, triagers should ignore assigned issue unless they are more than say 6 months old with no activity.
msg123004 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2010-12-01 18:46
Travis, feel free to close this without further action or use the information here to open a numpy issue.
History
Date User Action Args
2010-12-01 18:46:11belopolskysetstatus: open -> pending
assignee: belopolsky -> teoliphant
messages: + msg123004
2010-08-05 16:05:44belopolskysetnosy: + teoliphant
messages: + msg112992
2010-08-05 01:53:42terry.reedysetmessages: + msg112941
2010-08-04 22:33:25belopolskysetmessages: + msg112913
2010-08-04 22:26:15terry.reedysetnosy: + terry.reedy

messages: + msg112912
versions: - Python 2.6, Python 2.5
2010-07-20 06:27:11amaury.forgeotdarcsetmessages: + msg110870
2010-07-20 05:08:39belopolskysetmessages: + msg110865
2010-07-19 23:08:11amaury.forgeotdarcsetstatus: pending -> open
nosy: + amaury.forgeotdarc
messages: + msg110833

2010-07-19 17:55:29belopolskysetstatus: open -> pending
versions: + Python 2.7
resolution: invalid
messages: + msg110788

type: behavior
superseder: Py_TPFLAGS_LONG_SUBCLASS is not documented
stage: committed/rejected
2010-07-05 20:15:54belopolskysetmessages: + msg109352
2010-05-26 00:57:37belopolskysetassignee: belopolsky

nosy: + belopolsky
2009-05-18 09:25:27grubertsetnosy: + grubert

messages: + msg88022
versions: + Python 2.6
2009-03-11 16:06:13LambertDWsetnosy: + LambertDW
messages: + msg83471
2009-03-11 15:51:24barronhsetcomponents: + Library (Lib)
versions: + Python 2.5
2009-03-11 15:50:45barronhcreate