This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: SimpleHTTPServer throwed an exception due to negtive st_mtime attr in file
Type: behavior Stage: needs patch
Components: Library (Lib), Windows Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Sean.Wang, martin.panter, paul.moore, steve.dower, terry.reedy, tim.golden, xiang.zhang, zach.ware
Priority: normal Keywords:

Created on 2015-11-02 15:35 by Sean.Wang, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
test Sean.Wang, 2015-11-02 15:38 strange last modified date
Messages (8)
msg253926 - (view) Author: Sean Wang (Sean.Wang) Date: 2015-11-02 15:35
I transfered a file from remote Debian host to my local Windows 10 host using SecureFX.
I found that the file's last modifed date was ‎1900‎/‎1‎/1‎,‏‎0:00:00 on Windows.

I tried to serve this file to be downloaded, and it crashed as follows:
Exception happened during processing of request from ('192.168.1.102', 50978)

Traceback (most recent call last):
  File "C:\Python27\lib\SocketServer.py", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python27\lib\SocketServer.py", line 655, in __init__
    self.handle()
  File "C:\Python27\lib\BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "C:\Python27\lib\BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "C:\Python27\lib\SimpleHTTPServer.py", line 45, in do_GET
    f = self.send_head()
  File "C:\Python27\lib\SimpleHTTPServer.py", line 103, in send_head
    self.send_header("Last-Modified", self.date_time_string(fs.st_mtime))
  File "C:\Python27\lib\BaseHTTPServer.py", line 468, in date_time_string
    year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp)
ValueError: (22, 'Invalid argument')

I have checked the source code, and found it was because of the last modifed date of the file, I got this in console:

>>> os.fstat(f.fileno())
nt.stat_result(st_mode=33206, st_ino=4785074604093500L, st_dev=0L, st_nlink=1, st_uid=0, st_gid=0, st_size=3406L, st_atime=1446477520L, st_mtime=-2209017600L, st_ctime=1446370767L)

-2209017600L cannot be handled by "time.gmtime()" method and it throwed 
error
msg253927 - (view) Author: Sean Wang (Sean.Wang) Date: 2015-11-02 15:38
upload a sample test file
msg253948 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-02 22:37
I think your test file’s time is lost on the web server. On Linux it is pretty easy to make a file with an arbitrary time; maybe Windows has an equivalent:

$ touch -d "1 Jan 1900" test

I experimented with Wine, and it seems gmtime() raises ValueError on Python 2 and OSError on Python 3 for negative times, but under Linux negative times are handled successfully. (Wine seems to wrap the st_mtime field to a 32-bit unsigned value though, so I am unable to reproduce the original problem.)

The “time” module documentation says “The functions in this module may not handle dates and times before the epoch”, so maybe the HTTP module should be fixed to handle a ValueError (or OSError on Python 3).
msg254081 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2015-11-05 03:59
I'm afraid we can't say negative epoch is handled successfully on Linux.

Use $ touch -d "1 Jan 1900" test as a test, time.gmtime(os.fstat(f.fileno()).st_mtime) gives time.struct_time(tm_year=1899, tm_mon=12, tm_mday=31, tm_hour=15, tm_min=54, tm_sec=17, tm_wday=6, tm_yday=365, tm_isdst=0), which does not equals "1 Jan 1900". I think this is caused by python gmtime directly uses gmtime in C. Use the epoch of "1 Jan 1900" with C's gmtime does not give a right result. And while I am searching, I can not find any evidence that gmtime in C can give a right result with a negtive epoch.

And when I try the epoch of "1 Jan 1900" with "date -d @", it can generate the right result.
msg254082 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-05 04:15
Perhaps you have a timezone set (I don’t). This might work better:

touch -d "1 Jan 1900 UTC" test

My thinking is if os.stat() returns a negative timestamp from the OS (GNU C library in the case of Linux), it may be worth trying gmtime() on that value since it also calls the OS (glibc) function.
msg254083 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2015-11-05 04:44
You're right. Actually I do think of timezone when I do the experiment. But I think different timezone will only affect the hour. It turns out I should learn more about time.
msg254246 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-07 00:50
Does this affect 3.x also?  I would expect that it does.

The question for this issue is whether the program should stop on a gmtime error and say "I will not serve this file until you fix the modification time." or whether it should catch and workaround the problem and merely warn about the mtime.

For the latter, BaseHTTPServer could catch the gmtime error and use the current time instead.  Or SimpleHTTPServer could catch the exception and omit the Last Modified: header, if that is allowed.  I believe the purpose of this header is for caching, and the current time would say to replace any cached value.
msg254249 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-07 01:41
I assume it affects Python 3, though I suspect the exception is OSError, not ValueError. But it would be good if someone with Windows (or other affected OS) could confirm.

I think the server should serve the file, with just a best-effort attempt to serve the timestamp. Some other options:

* Maybe using datetime rather than the OS’s gmtime() would be more reliable
* Omit the Last-Modified header if the timestamp cannot be represented
* Make time.gmtime() more platform-independent (probably against the original spirit of the module)

I don’t think setting Last-Modified to the current time is a particularly good idea. I guess omitting the field would have a similar effect on caching, without actually serving a misleading value.
History
Date User Action Args
2022-04-11 14:58:23adminsetgithub: 69720
2015-11-07 01:41:26martin.pantersetmessages: + msg254249
versions: + Python 3.4, Python 3.5, Python 3.6
2015-11-07 00:50:51terry.reedysetnosy: + terry.reedy
messages: + msg254246
2015-11-05 04:44:05xiang.zhangsetmessages: + msg254083
2015-11-05 04:15:11martin.pantersetmessages: + msg254082
2015-11-05 03:59:18xiang.zhangsetnosy: + xiang.zhang
messages: + msg254081
2015-11-02 22:38:00martin.pantersetnosy: + paul.moore, tim.golden, martin.panter, zach.ware, steve.dower
messages: + msg253948

components: + Windows
type: crash -> behavior
stage: needs patch
2015-11-02 15:38:20Sean.Wangsetfiles: + test

messages: + msg253927
2015-11-02 15:35:17Sean.Wangcreate