msg200088 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-16 21:43 |
zipfile.PyZipFile needs a filter function.
Reason:
When creating an archive of the python lib, we don't want the tests.
Especially the test file "badsyntax_future3.py" does not compile.
Use case:
With this little addition, it becomes very easy to create a zip file
of the whole python library. See the attached use case.
|
msg200089 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-16 21:59 |
Here is my use case as an example.
With this patch above, I can easily create a .zip file of the standard lib.
This was no longer possible at all, after revision 17558,
from 2001-04-18:
"""This is a test"""
from __future__ import nested_scopes
from __future__ import rested_snopes
def f(x):
def g(y):
return x + y
return g
print f(2)(4)
|
msg200112 - (view) |
Author: Georg Brandl (georg.brandl) *  |
Date: 2013-10-17 06:21 |
Looks fine but -- as a new feature -- is 3.4 only, and needs docs and tests.
|
msg200269 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-18 15:04 |
Hi Georg,
So do you think it is ok this way?
I was not sure if extending the function with an optional
arg is ok, or if a method to configure PyZipFile would be better.
At the moment I just needed the simple functionality.
Should it maybe get a regex like compileall.py ?
And a general question:
What is the right constraint for a filter function?
As I wrote it, returning nothing would simply be treated as "False".
Maybe it is better to enforce a return value which explicitly forbids
to be just None, which often just means "I forgot the return value" ?
About feature or fix: Well, I need this for python2.7, because without
it, the whole purpose of PyZipFile is pretty questionable. I might argue
it a fix, because it crashes on the standard library ;-)
Anyway, tell me and I'l add test, docs and put it into dev.
cheers - chris
|
msg200275 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2013-10-18 15:33 |
I don't think this is needed. You can walk a tree and call writepy() for files and directories which you want.
|
msg200279 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-18 15:56 |
@serhiy.storchaka
> I don't think this is needed. You can walk a tree and call writepy()
> for files and directories which you want.
What exactly do mean by "this" and "needed"?
I cannot see the connection of my initial post and your reply.
Running PyZipFile on a package dir of the standard lib _does_ traverse
the tree, and there is no way to stop it from doing that.
That was the whole point of the issue.
Please correct me if I'm missing something.
|
msg200282 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2013-10-18 16:34 |
"this" is a filter function. "Not needed" means that you can got what you want without adding a filter function to zipfile.PyZipFile. Just don't call writepy() on directories which contains files which shouldn't be zipped.
|
msg200290 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-18 17:24 |
Ah, I understand:
The case that does not compile comes from the toplevel "test" folder,
which I could have excluded explicitly.
But it is not a complete solution:
If I want to add every package from the standard lib, then I necessarily
encounter enclosed test folders, for instance:
Lib/unittest
contains
Lib/unittest/test
Your hint to just not call writepy() on directories which contain files
which shouldn't be zipped means that I cannot use writepy at all,
because many library packages contain tests.
But that is the only purpose of class PyZipFile, therefore the patch.
|
msg200684 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2013-10-21 01:59 |
New changeset 34fb83421119 by Christian Tismer in branch 'default':
add a filterfunc to zip file.PyZipFile.writepy, issue 19274
http://hg.python.org/cpython/rev/34fb83421119
|
msg200701 - (view) |
Author: Georg Brandl (georg.brandl) *  |
Date: 2013-10-21 06:26 |
Hi Chris, your commit is a bit hard to review due to all the unrelated spacing changes. I assume this is done automatically by your editor? It's probably best to switch off that feature for CPython development :)
|
msg200702 - (view) |
Author: Georg Brandl (georg.brandl) *  |
Date: 2013-10-21 06:38 |
While reviewing: is it intended that the filter is only called for directories and not for individual files?
|
msg200703 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2013-10-21 06:42 |
New changeset 2d39b3555951 by Georg Brandl in branch 'default':
#19274: use captured_stdout() in the test suite; add NEWS entry.
http://hg.python.org/cpython/rev/2d39b3555951
|
msg200750 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-21 11:39 |
ah, I just see that.
The problem was that the checkin drove me nuts. It forced me to
run reindent to normalize the code. I did that with my WindIde
editor, but this did not help.
The point was: Actually an end-of-line was missing at the end of
the files.
Sorry, I did not see that at all, because the indentation changes
are at the end. I usually avoid this strictly. It was just the check-in
rejection...
|
msg200856 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-22 00:28 |
@georg:
> While reviewing: is it intended that the filter is only called for directories and not for individual files?
Not really. I will add this, later. Just wanted to see if this makes
sense and it's worth the effort to extend it.
|
msg200859 - (view) |
Author: Christian Tismer (Christian.Tismer) *  |
Date: 2013-10-22 02:13 |
added that with tests.
|
msg212762 - (view) |
Author: R. David Murray (r.david.murray) *  |
Date: 2014-03-05 15:01 |
For future reference, the update Christian refers to in the previous message is 4f1121ae1cb5.
|
msg212810 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2014-03-06 15:09 |
New changeset 064ee489982e by R David Murray in branch 'default':
whatsnew: improve PyZipFile filterfuc entry, and its docs (#19274).
http://hg.python.org/cpython/rev/064ee489982e
|
|
Date |
User |
Action |
Args |
2022-04-11 14:57:52 | admin | set | github: 63473 |
2014-03-06 15:09:34 | python-dev | set | messages:
+ msg212810 |
2014-03-05 15:01:56 | r.david.murray | set | nosy:
+ r.david.murray messages:
+ msg212762
|
2013-10-22 02:13:11 | Christian.Tismer | set | messages:
+ msg200859 |
2013-10-22 00:28:31 | Christian.Tismer | set | messages:
+ msg200856 |
2013-10-21 11:39:20 | Christian.Tismer | set | messages:
+ msg200750 |
2013-10-21 06:42:19 | python-dev | set | messages:
+ msg200703 |
2013-10-21 06:38:51 | georg.brandl | set | messages:
+ msg200702 |
2013-10-21 06:26:15 | georg.brandl | set | messages:
+ msg200701 |
2013-10-21 02:01:06 | Christian.Tismer | set | status: open -> closed |
2013-10-21 01:59:55 | python-dev | set | nosy:
+ python-dev messages:
+ msg200684
|
2013-10-18 17:24:55 | Christian.Tismer | set | messages:
+ msg200290 |
2013-10-18 16:34:19 | serhiy.storchaka | set | messages:
+ msg200282 |
2013-10-18 15:56:02 | Christian.Tismer | set | messages:
+ msg200279 |
2013-10-18 15:33:43 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages:
+ msg200275
|
2013-10-18 15:04:57 | Christian.Tismer | set | messages:
+ msg200269 |
2013-10-17 06:21:21 | georg.brandl | set | versions:
+ Python 3.4, - Python 2.7 nosy:
+ georg.brandl
messages:
+ msg200112
stage: patch review |
2013-10-16 21:59:41 | Christian.Tismer | set | files:
+ make_libzip.py
messages:
+ msg200089 |
2013-10-16 21:43:52 | Christian.Tismer | create | |