classification
Title: zipfile: Bad error message when zipping a file with timestamp before 1980
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ced, cmcqueen1975, ezio.melotti, ivb, loewis, mark.dickinson, orsenthil, petri.lehtinen, python-dev, r.david.murray, techtonik
Priority: normal Keywords: needs review, patch

Created on 2009-05-22 16:50 by ivb, last changed 2017-08-16 02:17 by cmcqueen1975. This issue is now closed.

Files
File name Uploaded Description Edit
test6090.py techtonik, 2010-04-06 07:54
zipfile_timestamps_before_1980.patch petri.lehtinen, 2011-07-06 20:29
Messages (9)
msg88201 - (view) Author: Ivan Bykov (ivb) Date: 2009-05-22 16:50
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit 
(Intel)] on win32
IDLE 2.6.2      
>>> import zipfile
>>> new = zipfile.ZipFile('C:\\x', 'w', zipfile.ZIP_DEFLATED)
>>> zi = zipfile.ZipInfo('test',(1,2,3,4,5,6))
>>> new.writestr(zi,'fgh')

Warning (from warnings module):
  File "H:\programs\python\lib\zipfile.py", line 1105
    self.fp.write(zinfo.FileHeader())
DeprecationWarning: struct integer overflow masking is deprecated

Warning (from warnings module):
  File "H:\programs\python\lib\zipfile.py", line 1105
    self.fp.write(zinfo.FileHeader())
DeprecationWarning: 'H' format requires 0 <= number <= 65535
>>>
msg102437 - (view) Author: anatoly techtonik (techtonik) Date: 2010-04-06 07:54
This code is broken in 2.7alpha4 - it doesn't add file at all.

Traceback (most recent call last):
  File "test.py", line 5, in <module>
    new.writestr(zi,'fgh')
  File "C:\~env\Python27\lib\zipfile.py", line 1099, in writestr
    self.fp.write(zinfo.FileHeader())
  File "C:\~env\Python27\lib\zipfile.py", line 342, in FileHeader
    len(filename), len(extra))
struct.error: ushort format requires 0 <= number <= USHRT_MAX
msg102461 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-04-06 12:33
Making the stage test needed, since we need to get the test into the test suite as a unit test.
msg102534 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-04-07 11:14
This is not a bug in the code, but in the application. You are passing a date before 1980; this is not supported in zipfiles. Try passing 1980 instead of 1.

I think the error message could be better, though; it should probably be a ValueError.

Reducing the priority to normal.
msg103024 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-04-13 09:13
[Martin]
> I think the error message could be better, though; it should probably be a ValueError.

Do you mean a ValueError at the level of the struct module, or the zipfile module.

I'd quite like to change all the exceptions raised by the struct module to ValueError or TypeError (as appropriate);  I'm not sure what the point of struct.error is supposed to be.  I've resisted making this change up until now for backwards compatibility reasons, but perhaps it could be considered for 3.2 (but not for 2.7).
msg139950 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2011-07-06 20:29
Retitled to reflect that the error message should be enhanced.

Attached a patch for 2.7 that raises ValueError for timestamps before 1980, documents that 1980 or later is required, and adds some tests.
msg145925 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-10-19 17:53
New changeset 649ac338203f by Senthil Kumaran in branch '2.7':
Fix closes Issue6090 - Raise a ValueError, instead of failing with unrelated
http://hg.python.org/cpython/rev/649ac338203f

New changeset 12f3e86e9041 by Senthil Kumaran in branch '3.2':
3.2 - Fix closes Issue6090 - Raise a ValueError, instead of failing with unrelated
http://hg.python.org/cpython/rev/12f3e86e9041

New changeset 55318658e1be by Senthil Kumaran in branch 'default':
default - Fix closes Issue6090 - Raise a ValueError, instead of failing with unrelated
http://hg.python.org/cpython/rev/55318658e1be
msg145927 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-10-19 17:54
Fixed in all active branches. Thanks for the patch, Petri. 
Mark, for this issue, raising ValueError from zipfile was seemingly a  right thing to do, the previous error from struct for a side effect of sending a value lower than 1980.
msg300323 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2017-08-16 02:17
One ongoing weakness I see with this situation is that it's difficult to code a suitable work-around if a user wants to zip files that have a date < 1980 (e.g. to zip it with a datestamp of 1-Jan-1980).
https://stackoverflow.com/q/45703747/60075

I am trying to create a zip file with Python 3.5.2 zipfile, on Linux. Some of the files I'm trying to add have timestamps of 1-Jan-1970 (embedded system without a real-time clock module). So zipfile gives an exception:

ValueError: ZIP does not support timestamps before 1980

My goal then is to implement a work-around to add these files to the zip file with a timestamp of 1-Jan-1980. However, I am finding it difficult to find a suitable work-around.

At first I thought I can do this:

def zinfo_from_file(fullname, arcname):
    st = os.stat(fullname)
    mtime = time.localtime(st.st_mtime)
    date_time = mtime[0:6]
    if date_time[0] < 1980:
        date_time = (1980, 1, 1, 0, 0, 0)
    zinfo = zipfile.ZipInfo(arcname, date_time)
    return zinfo

...
zinfo = zinfo_from_file(fullname, arcname)
chunksize=512
with open(fullname, 'rb') as src, myzipfile.open(zinfo, 'w') as dest:
    while True:
        data = src.read(chunksize)
        if not data:
            break
        dest.write(data)
...

However, it turns out that myzipfile.open(zinfo, 'w') is not supported until Python 3.6. (I'm using Yocto to build embedded Linux, which currently only supports Python 3.5.x.)

I guess I could try doing myzipfile.writestr(...), although then it appears that I have to load the entire file data into memory.
History
Date User Action Args
2017-08-16 02:17:23cmcqueen1975setnosy: + cmcqueen1975
messages: + msg300323
2011-10-19 17:54:46orsenthilsetnosy: + orsenthil
messages: + msg145927
2011-10-19 17:53:01python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg145925

resolution: fixed
stage: patch review -> resolved
2011-07-06 20:32:52petri.lehtinenlinkissue12198 superseder
2011-07-06 20:30:17petri.lehtinensettitle: zipfile: Bad error message when zipping a with timestamp before 1980 -> zipfile: Bad error message when zipping a file with timestamp before 1980
2011-07-06 20:29:46petri.lehtinensetfiles: + zipfile_timestamps_before_1980.patch


title: zipfile DeprecationWarning Python in 2.6, failure in 2.7 -> zipfile: Bad error message when zipping a with timestamp before 1980
keywords: + patch, needs review
nosy: + petri.lehtinen
versions: - Python 2.6
messages: + msg139950
stage: test needed -> patch review
2010-04-13 09:13:33mark.dickinsonsetnosy: + mark.dickinson
messages: + msg103024
2010-04-07 11:14:23loewissetpriority: critical -> normal
nosy: + loewis
messages: + msg102534

2010-04-06 14:32:57pitrousetpriority: normal -> critical
2010-04-06 12:33:53r.david.murraysetnosy: + r.david.murray
title: zipfile DeprecationWarning Python 2.5/2.6 -> zipfile DeprecationWarning Python in 2.6, failure in 2.7
messages: + msg102461

stage: needs patch -> test needed
2010-04-06 08:17:42ezio.melottisetversions: - Python 2.5
nosy: + ezio.melotti

priority: normal
type: behavior
stage: needs patch
2010-04-06 07:54:46techtoniksetfiles: + test6090.py


components: + Library (Lib), - Extension Modules
title: zipfile DeprecationWarning Python 2.6.2 -> zipfile DeprecationWarning Python 2.5/2.6
nosy: + techtonik
versions: + Python 2.5, Python 2.7
messages: + msg102437
2009-08-21 17:32:52cedsetnosy: + ced
2009-05-22 16:50:58ivbcreate