classification
Title: Inconsistencies with datetime.fromtimestamp(t) when t < 0
Type: behavior Stage:
Components: Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dobatymo, Paul Anton Letnes, Tiger-222, p-ganssle, steve.dower
Priority: normal Keywords:

Created on 2019-03-26 15:18 by Tiger-222, last changed 2020-10-14 01:11 by Dobatymo.

Messages (8)
msg338893 - (view) Author: Mickaël Schoentgen (Tiger-222) * Date: 2019-03-26 15:18
A similar issue was resolved with issue29097 (with 0 <= t <= 86399).

Here, we have an inconsistency between OSes when using datetime.fromtimestamp(t) when t < 0.
Tested on Python 3.6.7.

GNU/Linux:

>>> datetime.fromtimestamp(-1)
datetime.datetime(1970, 1, 1, 0, 59, 59)

macOS:

>>> datetime.fromtimestamp(-1)
datetime.datetime(1970, 1, 1, 0, 59, 59)

Windows (7 and 10):

>>> datetime.fromtimestamp(-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument


I think having a similar behavior between all Oses would be great, right?
msg338894 - (view) Author: Mickaël Schoentgen (Tiger-222) * Date: 2019-03-26 15:22
Reproduced on 3.6.8 and 3.7.3.
msg338896 - (view) Author: Mickaël Schoentgen (Tiger-222) * Date: 2019-03-26 15:24
And also reproduced on 3.8.0a3.
msg339038 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2019-03-28 13:41
From the documentation ( https://docs.python.org/3/library/datetime.html#datetime.datetime.fromtimestamp ):

> fromtimestamp() may raise OverflowError, if the timestamp is out of the range
> of values supported by the platform C localtime() or gmtime() functions, and
> OSError on localtime() or gmtime() failure. It’s common for this to be
> restricted to years in 1970 through 2038. Note that on non-POSIX systems that
> include leap seconds in their notion of a timestamp, leap seconds are ignored
> by fromtimestamp(), and then it’s possible to have two timestamps differing by
> a second that yield identical datetime objects. See also utcfromtimestamp().

So this is indeed the documented behavior. I agree that it would be good to unify the behavior across platforms if possible, but I think this would require us to have our own implementation of localtime() and/or gmtime().

That said, it might be possible to implement `fromtimestamp` with some equivalent of `datetime(1970, 1, 1) + timedelta(seconds=t)` on Windows when the value falls outside the accepted range. We'd probably need some better tests under different time zones to make sure that that would be acceptable.

I think it may be a good idea to change the targeted version to 3.8 or 3.9, because this is a change to the documented behavior of the function (albeit a desirable one that can probably be considered backwards compatible).
msg355822 - (view) Author: Paul Ganssle (p-ganssle) * (Python committer) Date: 2019-11-01 14:54
This has been coming up in a few different contexts lately, so I think it would be really good if we could get some sort of fix for it.

One option is to implement our own versions of these APIs for use in Windows, but a thought occurred to me recently: we have not investigated the possibility of seeing if Microsoft would be willing to either add support for negative timestamps in their localtime() or gmtime() implementations or add a new API that *does* support negative timestamps. It would also be good to rule out the possibility that such APIs already exist but we just don't know about them (preliminary googling doesn't show anything, though possibly something can be done with the Win32 APIs? Not sure how or if those work in C and how big a lift it would be to maintain compatibility if can switch: https://docs.microsoft.com/en-us/windows/win32/sysinfo/time-functions?redirectedfrom=MSDN ).

Adding Steve Dower to the nosy list in case he can shed some light onto the possibility of native support.
msg355831 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-11-01 18:11
I've emailed some colleagues to see what they can add here.

Ultimately, the Windows CRT is just doing arithmetic, since POSIX time formats do not resemble Windows time formats at all, so it's all emulation. So if we replaced it with our own calculation I don't think that would be the worst possible outcome.
msg371200 - (view) Author: Paul Anton Letnes (Paul Anton Letnes) Date: 2020-06-10 13:55
I've encountered an issue on anaconda python on windows 10 v1909 which I suspect is related. It looks like no dates in 1970 can be converted to datetime.timestamp():

Python 3.8.2 (default, Apr 14 2020, 19:01:40) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import datetime

In [2]: datetime.datetime(1970, 1, 2, 0, 0, 1, 123456).timestamp()
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-2-0ac473af013b> in <module>
----> 1 datetime.datetime(1970, 1, 2, 0, 0, 1, 123456).timestamp()

OSError: [Errno 22] Invalid argument

In [3]: datetime.datetime(1971, 1, 2, 0, 0, 1, 123456).timestamp()
Out[3]: 31618801.123456
msg378591 - (view) Author: Dobatymo (Dobatymo) Date: 2020-10-14 01:11
> I've encountered an issue on anaconda python on windows 10 v1909 which I suspect is related. It looks like no dates in 1970 can be converted to datetime.timestamp():

Yeah... there is more related weirdness going on.

>>> datetime(1970, 1, 3).astimezone(timezone.utc)
datetime.datetime(1970, 1, 2, 16, 0, tzinfo=datetime.timezone.utc)
>>> datetime(1970, 1, 2).astimezone(timezone.utc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
>>> datetime(1970, 1, 1, 16, 0, tzinfo=timezone.utc)
datetime.datetime(1970, 1, 1, 16, 0, tzinfo=datetime.timezone.utc)
History
Date User Action Args
2020-10-14 01:11:02Dobatymosetnosy: + Dobatymo
messages: + msg378591
2020-06-10 13:55:16Paul Anton Letnessetnosy: + Paul Anton Letnes
messages: + msg371200
2019-11-05 01:24:49vstinnersetnosy: - vstinner
2019-11-01 18:11:39steve.dowersetmessages: + msg355831
2019-11-01 14:54:46p-gansslesetnosy: + steve.dower
messages: + msg355822
2019-11-01 14:40:49p-gansslelinkissue38233 superseder
2019-03-28 13:41:47p-gansslesetmessages: + msg339038
2019-03-26 15:30:15Tiger-222settitle: OSes inconsistency whith datetime.fromtimestamp(t) when t < 0 -> Inconsistencies with datetime.fromtimestamp(t) when t < 0
2019-03-26 15:29:48Tiger-222settitle: [Windows] datetime.fromtimestamp(t) when t < 0 fails on Python 3.6 -> OSes inconsistency whith datetime.fromtimestamp(t) when t < 0
2019-03-26 15:24:53Tiger-222setmessages: + msg338896
versions: + Python 3.8
2019-03-26 15:23:53vstinnersetnosy: + p-ganssle
2019-03-26 15:23:40vstinnersetnosy: + vstinner
2019-03-26 15:22:17Tiger-222settype: behavior
messages: + msg338894
versions: + Python 3.7
2019-03-26 15:18:26Tiger-222create