classification
Title: calendar.timegm not always an inverse of time.gmtime
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: belopolsky, eitan.adler, martin.panter, tim.peters
Priority: normal Keywords:

Created on 2018-05-19 10:41 by eitan.adler, last changed 2018-05-20 19:25 by belopolsky.

Messages (8)
msg317111 - (view) Author: Eitan Adler (eitan.adler) * Date: 2018-05-19 10:41
How to reproduce:
∴cat bad.py
import time
import calendar

one = time.gmtime(1234567899)
two = calendar.timegm(time.gmtime(1234567899))
three = time.gmtime(two)

print(one)
print(two)
print(three)
print(one == three)


Expected behavior:
the functions behave as documented: they are inverses


Actual behavior:

∴/srv/src/python/cpython/python bad.py
time.struct_time(tm_year=2009, tm_mon=2, tm_mday=13, tm_hour=23, tm_min=31, tm_sec=15, tm_wday=4, tm_yday=44, tm_isdst=0)
1234567875
time.struct_time(tm_year=2009, tm_mon=2, tm_mday=13, tm_hour=23, tm_min=30, tm_sec=51, tm_wday=4, tm_yday=44, tm_isdst=0)
False

Details: python built from f65e31fee3 under debug mode
msg317133 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2018-05-19 18:51
They both look wrong to me.  Under 3.6.5 on Win10, `one` and `three` are the same.

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32

time.struct_time(tm_year=2009, tm_mon=2, tm_mday=13, tm_hour=23, tm_min=31, tm_sec=39, tm_wday=4, tm_yday=44, tm_isdst=0)

And that matches what `datetime` computes:

>>> from datetime import *
>>> datetime(1970, 1, 1) + timedelta(seconds=1234567899)
datetime.datetime(2009, 2, 13, 23, 31, 39)
msg317145 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2018-05-20 00:13
According to Wikipedia, there were 24 leap seconds before Feb 2009. So my guess is Eitan’s “gmtime” implementation is calculating the date as if the timestamp (1234567899) includes leap seconds, as in <https://en.wikipedia.org/wiki/Unix_time#TAI-based_variant>. But according to Posix, the calculation should be for exactly 86400 timestamp seconds per day, and it should not adjust for leap seconds.
msg317183 - (view) Author: Eitan Adler (eitan.adler) * Date: 2018-05-20 16:27
Thanks! I've confirmed this is the case on my system and will be starting a conversation about fixing it.

In the meantime I am leaving this bug open since we may want to detect this case in python and correct for it.
msg317190 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2018-05-20 16:57
I was able to reproduce this issue on Linux as follows

>>> import time, calendar, os
>>> os.environ['TZ'] = 'right/UTC'  # "right" timezones account for leap seconds
>>> time.tzset()
>>> calendar.timegm(time.gmtime(1234567899))
1234567875

This confirms Martin's insight that on the OP's system gmtime accounts for leap seconds.

There has been a general trend in Python to replace calls to system dependent functions in the time module with system independent reimplementations.  It is certainly possible to reimplement time.gmtime, but someone should champion this idea on python-dev.
msg317191 - (view) Author: Eitan Adler (eitan.adler) * Date: 2018-05-20 17:21
Relevant conversation: https://lists.freebsd.org/pipermail/freebsd-standards/2018-May/003714.html
msg317197 - (view) Author: Eitan Adler (eitan.adler) * Date: 2018-05-20 18:24
FTR this was entirely self-inflicted: I had a setting which explicitly asked for non-posix gmtime.

That said, leaving this open for the same reasons as above.
msg317199 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2018-05-20 19:25
Well, even in non-POSIX mode timegm and gmtime are the inverse of one another in BSD.  The problem that you see stems from the fact that time.gmtime is implemented as a call to the namesake C function while calendar.timegm is implemented in pure Python. It would be reasonable to bring some consistency to this situation.
History
Date User Action Args
2018-05-20 19:25:09belopolskysetmessages: + msg317199
2018-05-20 18:24:23eitan.adlersetmessages: + msg317197
2018-05-20 17:21:26eitan.adlersetmessages: + msg317191
2018-05-20 16:57:49belopolskysetversions: + Python 3.8
messages: + msg317190

assignee: belopolsky
type: behavior -> enhancement
stage: needs patch
2018-05-20 16:27:09eitan.adlersetmessages: + msg317183
2018-05-20 00:13:24martin.pantersetnosy: + martin.panter
messages: + msg317145
2018-05-19 18:51:16tim.peterssetnosy: + tim.peters
messages: + msg317133
2018-05-19 18:39:32serhiy.storchakasetnosy: + belopolsky
2018-05-19 10:42:40eitan.adlersettype: behavior
components: + Library (Lib)
2018-05-19 10:41:39eitan.adlercreate