zipfile.PyZipFile needs a filter function.

Reason:
When creating an archive of the python lib, we don't want the tests.
Especially the test file "badsyntax_future3.py" does not compile.

Use case:

With this little addition, it becomes very easy to create a zip file
of the whole python library. See the attached use case.

Here is my use case as an example.
With this patch above, I can easily create a .zip file of the standard lib.

This was no longer possible at all, after revision 17558,
from 2001-04-18:

"""This is a test"""
from __future__ import nested_scopes
from __future__ import rested_snopes

def f(x):
    def g(y):
        return x + y
    return g

print f(2)(4)

Looks fine but -- as a new feature -- is 3.4 only, and needs docs and tests.

Hi Georg,
So do you think it is ok this way?
I was not sure if extending the function with an optional
arg is ok, or if a method to configure PyZipFile would be better.
At the moment I just needed the simple functionality.
Should it maybe get a regex like compileall.py ?

And a general question:
What is the right constraint for a filter function?
As I wrote it, returning nothing would simply be treated as "False".
Maybe it is better to enforce a return value which explicitly forbids
to be just None, which often just means "I forgot the return value" ?

About feature or fix: Well, I need this for python2.7, because without
it, the whole purpose of PyZipFile is pretty questionable. I might argue
it a fix, because it crashes on the standard library ;-)

Anyway, tell me and I'l add test, docs and put it into dev.

cheers - chris

I don't think this is needed. You can walk a tree and call writepy() for files and directories which you want.

@serhiy.storchaka

I don't think this is needed. You can walk a tree and call writepy()
for files and directories which you want.

What exactly do mean by "this" and "needed"?
I cannot see the connection of my initial post and your reply.

Running PyZipFile on a package dir of the standard lib _does_ traverse
the tree, and there is no way to stop it from doing that.
That was the whole point of the issue.
Please correct me if I'm missing something.

"this" is a filter function. "Not needed" means that you can got what you want without adding a filter function to zipfile.PyZipFile. Just don't call writepy() on directories which contains files which shouldn't be zipped.

Ah, I understand:

The case that does not compile comes from the toplevel "test" folder,
which I could have excluded explicitly.

But it is not a complete solution:
If I want to add every package from the standard lib, then I necessarily
encounter enclosed test folders, for instance:

[Lib/unittest](https://github.com/python/cpython/blob/main/Lib/unittest)

contains

[Lib/unittest/test](https://github.com/python/cpython/blob/main/Lib/unittest/test)

Your hint to just not call writepy() on directories which contain files
which shouldn't be zipped means that I cannot use writepy at all,
because many library packages contain tests.
But that is the only purpose of class PyZipFile, therefore the patch.

New changeset 34fb83421119 by Christian Tismer in branch 'default':
add a filterfunc to zip file.PyZipFile.writepy, bpo-19274
http://hg.python.org/cpython/rev/34fb83421119

Hi Chris, your commit is a bit hard to review due to all the unrelated spacing changes. I assume this is done automatically by your editor? It's probably best to switch off that feature for CPython development :)

While reviewing: is it intended that the filter is only called for directories and not for individual files?

New changeset 2d39b3555951 by Georg Brandl in branch 'default':
bpo-19274: use captured_stdout() in the test suite; add NEWS entry.
http://hg.python.org/cpython/rev/2d39b3555951

ah, I just see that.
The problem was that the checkin drove me nuts. It forced me to
run reindent to normalize the code. I did that with my WindIde
editor, but this did not help.
The point was: Actually an end-of-line was missing at the end of
the files.
Sorry, I did not see that at all, because the indentation changes
are at the end. I usually avoid this strictly. It was just the check-in
rejection...

@georg:

While reviewing: is it intended that the filter is only called for directories and not for individual files?

Not really. I will add this, later. Just wanted to see if this makes
sense and it's worth the effort to extend it.

added that with tests.

For future reference, the update Christian refers to in the previous message is 4f1121ae1cb5.

New changeset 064ee489982e by R David Murray in branch 'default':
whatsnew: improve PyZipFile filterfuc entry, and its docs (bpo-19274).
http://hg.python.org/cpython/rev/064ee489982e