New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Calling timestamp() on a datetime object modifies the timestamp of a different datetime object. #66817
Comments
I have an example script here[1]. |
This has nothing to do with the datetime module. Attached script reduces the issue to a bug (?) in time.mktime() with the corresponding timezone. time.mktime() is a thin wrapper around the C library's mktime() function, so it is probably not a bug in Python at all. Note your script is fixed by removing ".replace(tzinfo=None)". It seems conter-productive to take the pain to create an aware timezone and then make it naive, IMHO. datetime.timestamp() falls back on time.mktime() when the datetime is naive (i.e. it asks the OS to do the computation). (I did my tests under Ubuntu 13.10, btw) |
Hi Antoine, thanks for taking a look. I should explain further. This code is for an introspection tool[1] that provides an interface to write tests in python against an application. This code is a workaround for using large timestamps[2] (i.e. larger than 32bit time_t) the original suggested code (from SO, using timedelta) has an inconsistancy with the hour in some cases (this example in the NZ timezone):
>>> datetime.datetime.fromtimestamp(2047570047)
datetime.datetime(2034, 11, 19, 17, 27, 27)
>>> datetime.datetime.fromtimestamp(0) + datetime.timedelta(seconds=2047570047)
datetime.datetime(2034, 11, 19, 18, 27, 27) The code you see here in my example, creating a timezone aware object, is a result of 'fixing' this behaviour I've seen. [1] https://launchpad.net/autopilot |
In general, you cannot expect datetime.fromtimestamp(0) + timedelta(seconds) to return the same value as datetime.fromtimestamp(seconds). This will only be true if you are lucky enough to live in an area where local government did not mess with timezones since 1970. In your particular case, I think what is happening is that you have DST in January, but you did not have that on January 1, 1970. $ zdump -v NZ | grep NZDT | head
NZ Sat Nov 2 14:00:00 1974 UTC = Sun Nov 3 03:00:00 1974 NZDT isdst=1
NZ Sat Feb 22 13:59:59 1975 UTC = Sun Feb 23 02:59:59 1975 NZDT isdst=1
NZ Sat Oct 25 14:00:00 1975 UTC = Sun Oct 26 03:00:00 1975 NZDT isdst=1
NZ Sat Mar 6 13:59:59 1976 UTC = Sun Mar 7 02:59:59 1976 NZDT isdst=1
NZ Sat Oct 30 14:00:00 1976 UTC = Sun Oct 31 03:00:00 1976 NZDT isdst=1
NZ Sat Mar 5 13:59:59 1977 UTC = Sun Mar 6 02:59:59 1977 NZDT isdst=1
NZ Sat Oct 29 14:00:00 1977 UTC = Sun Oct 30 03:00:00 1977 NZDT isdst=1
NZ Sat Mar 4 13:59:59 1978 UTC = Sun Mar 5 02:59:59 1978 NZDT isdst=1
NZ Sat Oct 28 14:00:00 1978 UTC = Sun Oct 29 03:00:00 1978 NZDT isdst=1
NZ Sat Mar 3 13:59:59 1979 UTC = Sun Mar 4 02:59:59 1979 NZDT isdst=1 |
Antoine, I don't think the behavior that you have shown is a bug in strict sense. On my Mac, I get $ python mkbug.py breakme
1396702800.0
1396702800.0 but on Linux, $ python mkbug.py breakme
1396706400.0
1396702800.0 The problem here is that time.mktime((2014, 4, 6, 2, 0, 0, -1, -1, -1)) $ TZ=NZ date -d @1396706400
Sun Apr 6 02:00:00 NZST 2014
$ TZ=NZ date -d @1396702800
Sun Apr 6 02:00:00 NZDT 2014 It is unfortunate that Linux C library (glibc?) makes different guess at different time, but I don't think this violates any applicable standards. |
Alexander, Ah ok thanks for clarifying that. Am I wrong then to think that this code[1] should work as I think it should (i.e. datetime_from_large_timestamp(example_ts) == datetime.fromtimestamp(example_ts)) I'm trying to be able to handle timestamps larger than the 32bit time_t limit, which is why I'm doing these gymnastic steps. |
Your code as good as your timezone library, but you should realize that by discarding tzinfo you are making your "local_stamp" ambiguous. I am not familiar with dateutil.tz, but pytz I believe uses some tricks to make sure astimezone() result remembers the isdst flag. If you API requires naive local datetime objects, you need to carry isdst flag separately if you want to disambiguate between 2014-04-06 02:00 NZST and 2014-04-06 02:00 NZDT. On top of that, you will not be able to use datetime.timestamp() method and will have to use time.mktime or whatever equivalent utility your timezone library provides. Note that I was against adding datetime.timestamp() for this specific reason: it is supposed to be inverse of datetime.fromtimestamp(), but since the later is not monotonic, no such inverse exists in the strict mathematical sense. See msg133039 in bpo-2736. BTW, if you open a feature request to add isdst=-1 optional argument to datetime.timestamp(), you will have my +1. |
Christopher, About your script http://paste.ubuntu.com/8562027/ dateutil may break if the local timezone had different UTC offset in the past. To get the correct time for 1414274400 timezone in Europe/Moscow timezone, >>> import time
>>> import os
>>> os.environ['TZ'] = 'Europe/Moscow'
>>> time.tzset()
>>> from datetime import datetime, timezone
>>> from tzlocal import get_localzone
>>> datetime.fromtimestamp(1414274400, get_localzone())
datetime.datetime(2014, 10, 26, 1, 0, tzinfo=<DstTzInfo 'Europe/Moscow' MSK+3:00:00 STD>)
>>> datetime.utcfromtimestamp(1414274400).replace(tzinfo=timezone.utc).astimezone(get_localzone())
datetime.datetime(2014, 10, 26, 1, 0, tzinfo=<DstTzInfo 'Europe/Moscow' MSK+3:00:00 STD>)
>>> datetime.fromtimestamp(1414274400) # wrong
datetime.datetime(2014, 10, 26, 2, 0)
>>> datetime.fromtimestamp(1414274400, timezone.utc).astimezone() # wrong
datetime.datetime(2014, 10, 26, 2, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 14400), 'MSK'))
>>> datetime.utcfromtimestamp(1414274400).replace(tzinfo=timezone.utc).astimezone() # wrong
datetime.datetime(2014, 10, 26, 2, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 14400), 'MSK'))
>>> from dateutil.tz import gettz, tzutc
>>> datetime.fromtimestamp(1414274400, gettz()) # wrong
datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow'))
>>> datetime.fromtimestamp(1414274400, tzutc()).astimezone(gettz()) # wrong
datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow'))
>>> datetime.utcfromtimestamp(1414274400).replace(tzinfo=tzutc()).astimezone(gettz()) # wrong
datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow')) To avoid surprises, always use UTC time to perform date arithmetics: EPOCH = datetime(1970, 1,1, tzinfo=pytz.utc)
utc_dt = EPOCH + timedelta(seconds=timestamp) should work for dates after 2038 too. To convert it to the local timezone: from tzlocal import get_localzone
local_dt = utc_dt.astimezone(get_localzone())
ts = (local_dt - EPOCH) // timedelta(seconds=1)
assert ts == timestamp # if timestamp is an integer Python stdlib assumes POSIX encoding for time.time() value (bpo-22356) |
Thanks Akira, everyone for all the info. It looks like I've highjacked this bug comments to trying to solve my first problem (i.e. datetime objects for large timestamps) instead of the bug at hand, I feel should move that conversation elsewhere. It appears from Alexander's comment that it might not be a bug. I need to figure out if I need to work around this or use some other mechanism. |
Christopher, I've already pointed out a fix in another message: just remove ".replace(tzinfo=None)". Doing computations on UTC datetimes (rather than naive) should ensure you don't encounter any ambiguities. |
I am reopening this issue as an "enhancement" because I would like to revisit it in light of PEP-495. |
Superseded by bpo-28067. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: