classification
Title: python3 fromtimestamp generates OSError
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: SilentGhost, belopolsky, jamercee, p-ganssle
Priority: normal Keywords:

Created on 2020-05-25 16:47 by jamercee, last changed 2020-05-26 17:33 by SilentGhost.

Messages (4)
msg369895 - (view) Author: Jim Carroll (jamercee) * Date: 2020-05-25 16:47
We encountered an interesting mtime problem in the field, that I believe represents a bug in python's datetime timestamp handling.

A file stored on a windows server had the last-modified date '1/1/4501' (that's the year 4501). os.path.getmtime() returns a valid timestamp, but when we try to pass this back into datetime.datetime.fromtimestamp() we get an OSError.

I understand that generating an OSError when the date exceeds the epoch support on Windows is consistent with the python docs. In our case, the date is clearly supported by Windows as evidenced by it's storage in the filesystem. Further, we can reproduce the situation using the cygwin touch utility.

>>> import os, datetime
>>> os.system('touch -d "4501-01-01" file.txt')
>>> t = os.path.getmtime('file.txt')
>>> datetime.datetime.fromtimestamp(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument

What's interesting is we can manually convert it with reference to the epoch

>>> datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=t)
datetime.datetime(4501, 1, 1, 5, 0)

We used Windows 10-Pro for our tests running python 3.8.1.
msg369903 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2020-05-25 18:18
You seem to have read the docs, so I'm a bit confused why you think that is a bug. According to https://docs.python.org/3/library/datetime.html#datetime.datetime.fromtimestamp

> fromtimestamp() may raise OverflowError, if the timestamp is out of the range of values supported by the platform C localtime() or gmtime() functions, and OSError on localtime() or gmtime() failure.

This looks exactly like this type of error. It is completely irrelevant whether Windows supports a timestamp this far in the future, clearly platform's localtime() / gmtime() do not. Constructing datetime object in Python does not require such system calls and their limitations no longer apply.
msg370011 - (view) Author: Jim Carroll (jamercee) * Date: 2020-05-26 16:24
My bad. I read the docs, but mistakenly believed platform support meant OS. I figured since Windows maketh then Windows should taketh. I've spent the day studying the _datetimemodule.c code and now realize my error.

Question -- it seems to me an unnecessary limitation.  If someone were willing to submit a platform neutral version of localtime/gmtime that worked identically on all platforms, would this be met with eagerness? Or more of a shoulder shrug?

It's a fair amount of effort, and I'm willing to commit to it, but if the issue only matters to me, I can work around it.
msg370023 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2020-05-26 17:33
I think I would have to leave judgement on that to core developers. This, if accepted, would be a candidate for 3.10.
History
Date User Action Args
2020-05-26 17:33:05SilentGhostsetmessages: + msg370023
versions: + Python 3.10, - Python 3.8
2020-05-26 16:24:34jamerceesetmessages: + msg370011
2020-05-25 18:18:28SilentGhostsetnosy: + SilentGhost, p-ganssle, belopolsky
type: behavior
messages: + msg369903
2020-05-25 16:47:53jamerceecreate