Title: ZIP does not support timestamps before 1980
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Dormouse759, petr.viktorin, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2018-07-11 14:35 by petr.viktorin, last changed 2018-07-13 13:05 by petr.viktorin.

Pull Requests
URL Status Linked Edit
PR 8270 open Dormouse759, 2018-07-13 11:43
Messages (7)
msg321460 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2018-07-11 14:35
The ZIP format cannot handle times before 1980.
Issue6090 provided a nice error message for trying to add such files.

I'm seeing a system for reproducible builds that sets mtime to 1970 (zero UNIX timestamp), resulting in files that Python can't package into (zip-based) wheels.

At least here on Fedora, the `zip` command-line utility silently bumps old timestamps to 1980-01-01. Of course, silently corrupting data would not be good default behavior for Python.
But in many cases timestamps don't matter. It would be nice to give ZipFile and ZipFile.write() a `strict_timestamps=True` keyword argument that could be turned off.
msg321464 - (view) Author: Marcel Plch (Dormouse759) * Date: 2018-07-11 14:40
I'm going to have a closer look at this and try to add the option.
msg321595 - (view) Author: Marcel Plch (Dormouse759) * Date: 2018-07-13 11:45
I have created a PR for this:
msg321597 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-13 12:04
There are ZIP extensions which allow to save timestamps before 1980 and with better than 2-seconds resolution. It would be better to use this feature. But first we need to resolve issue17681.
msg321598 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2018-07-13 12:22
I'm not sure the extensions will solve this problem fully.
If an implementation that doesn't support these extensions, how does it handle them?

For example, if Python 3.8 implements the extensions, I use py3.8 to create a zipfile containing old timestamps, and then want to uncompress the file using Python 2.7, how do I avoid losing the timestamp information?
msg321600 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-13 12:37
You can't, because Python 2.7 doesn't support it. But you will be able to pack files in a ZIP archive and extract them without a loss on 3.8 and with a loss of a timestamp on 2.7. And you will be able to use a third-party utilities for extracting files without a loss.
msg321607 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2018-07-13 13:05
Which third-party utilities support these? As I said, AFAIK `zip` on my system does not.

I assume the loss of data is the reason we have an error now -- if that wasn't a concern, zipfile could just silently bump the timestamp to 1980.

When the extensions are implemented, `strict_timestamps=False` should *additionally* use the extension, but default should still be to raise the error (to prevent metadata loss when decompressing with older/simpler tools).
Date User Action Args
2018-07-13 13:05:01petr.viktorinsetmessages: + msg321607
2018-07-13 12:37:48serhiy.storchakasetmessages: + msg321600
2018-07-13 12:22:14petr.viktorinsetmessages: + msg321598
2018-07-13 12:04:25serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg321597
2018-07-13 11:45:19Dormouse759setmessages: + msg321595
2018-07-13 11:43:45Dormouse759setkeywords: + patch
stage: patch review
pull_requests: + pull_request7805
2018-07-11 14:40:11Dormouse759setnosy: + Dormouse759
messages: + msg321464
2018-07-11 14:35:58petr.viktorincreate