Currently, ZipFile only accepts ascii or utf8 as file
name encodings. On Windows (Japanese), usually CP932
is used for it. So currently, when we melt ZipFile
via py3k, non-ascii file name becomes strange. Can we handle
this issue? (ie: adding encoding option for ZipFile#__init__)

The ZIP format specification mentions only cp437 and utf8: http://www.pkware.com/documents/casestudies/APPNOTE.TXT see Apeendix D.
Do zip files created on Japanese Windows contain some information about the encoding they use?
Or do some programs write cp932 where they are supposed to use one of the encodings above?

No, there is no indication in the zipfile that it deviates from the spec. That doesn't stop people from creating such zipfiles, anyway; many zip tools ignore the spec and use instead CP_ACP (which, of course, will then get misinterpreted if extracted on a different system).

I think we must support this case somehow, but must be careful to avoid creating such files unless explicitly requested. One approach might be to have two encodings given: one to interpret the existing filenames, and one to be used for new filenames (with a recommendation to never use that parameter since zip now supports UTF-8 in a well-defined manner).

@Hirokazu: Can you attach a small test archive?

Yes, we can add a "default_encoding" attribute to ZipFile and add an optional default_encoding argument to its constructor.

I'm not sure why, but I got BadZipFile error now. Anyway,
here is cp932 zip file to be created with python2.7.

In #10972, I propose to add an option for the filename encoding to UTF-8. But I would like to force UTF-8 to create a ZIP file, it doesn't concern the decompression of a ZIP file.

Proposal of a specification to fix both issues at the same time.


"default_encoding" name is confusing because it doesn't specify if it is the encoding of (text?) file content or the encoding the filename. Why not simply "filename_encoding"?

The option can be added in multiple places:
 - argument to ZipFile constructor: this is needed to decompress
 - argument to ZipFile.write() and ZipInfo, because they are 3 different manners to add files

ZipFile.filename_encoding (and ZipInfo.filename_encoding) will be None by default: in this case, use the current algorithm (try cp437 or use UTF-8). Otherwise, use the encoding. If the encoding is UTF-8: set unicode flag.

Examples:
---
zipfile.ZipFile("non-ascii-cp932.zip", filename_encoding="cp932")

f = zipfile.ZipFile("test.zip", "w")
f.write(filename, filename_encoding="UTF-8")
info = ZipInfo(filename, filename_encoding="UTF-8")
f.writestr(info, b'data')
---

Don't add filename_encoding argument to ZipFile.writestr(), because it may conflict if a ZipInfo is passed and ZipInfo.filename_encoding and filename_encoding are different.

I closed issue #12048 as a duplicate of this issue: yaoyu wants to uncompress a ZIP file having filenames encoded to GBK.

I fixed this problem.
I make new methos _decode_filename().

umedoblock: your patch is incorrect, as it produces moji-bake. if there is a file name b'f\x94n', it will decode as sjis under your patch (to u'f\u99ac'), even though it was meant as cp437 (i.e. u'f\xf6n').

Hi, Martin.
I tried your test case with attached file.
And I got below result.

p3 ./encodings.py
encoding: sjis, filename: f馬
encoding: cp437, filename: fön
sjis_filename = f馬
cp437_filename = fön

There are two success cases.
So I think that the patch needs to change default_encoding
before or in _decode_filename().

But I have no idea about how to change a default_encoding.