classification
Title: make clean does not remove pickle files
Type: behavior Stage: resolved
Components: 2to3 (2.x to 3.x conversion tool), Build Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: benjamin.peterson, drukker, mark.dickinson
Priority: low Keywords: patch

Created on 2009-09-23 20:31 by drukker, last changed 2009-09-24 19:28 by mark.dickinson. This issue is now closed.

Files
File name Uploaded Description Edit
clean-pickles.patch drukker, 2009-09-23 20:31 add pickleremoval target to Makefile.pre.in
clean-grammar-pickles.patch drukker, 2009-09-24 10:09
Messages (7)
msg93049 - (view) Author: Derk Drukker (drukker) Date: 2009-09-23 20:31
make clean and make distclean don't remove the grammar pickles generated
by load_grammar in Lib/lib2to3/pgen2/driver.py
(Lib/lib2to3/Grammar3.2.0.alpha.0.pickle,
Lib/lib2to3/PatternGrammar3.2.0.alpha.0.pickle).

Proposed patch attached. It removes all *.pickle files, which I assume
is safe. (Patch also corrects a comment.)
msg93055 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 07:11
I don't think it is safe to remove all *.pickle files in the source 
directory;  even if there aren't any other useful .pickle files right now, 
there could be in the future (e.g. for pickle testing).

I'd prefer something more targeted.

Are these .pickle files generated during the build process, or just during 
testing?  Maybe it should be the responsibility of the 2to3 tests to clean 
up after themselves?
msg93058 - (view) Author: Derk Drukker (drukker) Date: 2009-09-24 08:39
These .pickle files aren't created by the tests themselves, but they do
show up after running 'make test', or more specifically after running
'./python Lib/test/regrtest.py -v test_lib2to3'.

This is because a grammar generated from a .txt grammar file is cached
as a pickle for faster access by load_grammar in
Lib/lib2to3/pgen2/driver.py. A pickle file then ends up in the same
directory as the .txt file.

IMHO, though it may be preferable to remove just the pickle files in
Lib/lib2to3, it should never be required for tests to have pickle files
already available in the source tree. Instead, the test should create a
temporary pickle file, e.g. from a bytes object. Furthermore, binary
files shouldn't be versioned without good reason. Thus, removing all
pickle files should be safe for the future.
msg93061 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 09:34
> IMHO, though it may be preferable to remove just the pickle files in
> Lib/lib2to3,

Sounds reasonable.

> it should never be required for tests to have pickle files
> already available in the source tree.

Well, I don't know about 'should', but the current tests already do
make use of pickle files in the source tree:  see Lib/test/*.pck.
So there are already versioned pickle files (but not .pickle files :-).
msg93062 - (view) Author: Derk Drukker (drukker) Date: 2009-09-24 10:09
You are right. Guess I was being a little too dogmatic. :-)

I hadn't found those .pck files, because I was only looking for binary
files.

Here's a new patch proposal.
msg93067 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 11:14
Thanks!  I'll apply this later today, unless Benjamin gets there first. :)
msg93089 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-09-24 19:28
Committed in r75047 (trunk).  Merged in r75048 (release26-maint), r75049 
(py3k) and r75050 (release31-maint).
History
Date User Action Args
2009-09-24 19:28:43mark.dickinsonsetstatus: open -> closed
assignee: mark.dickinson
messages: + msg93089

stage: patch review -> resolved
2009-09-24 11:14:42mark.dickinsonsetresolution: accepted
messages: + msg93067
2009-09-24 10:09:30drukkersetfiles: + clean-grammar-pickles.patch

messages: + msg93062
2009-09-24 09:34:54mark.dickinsonsetmessages: + msg93061
versions: + Python 2.6, Python 2.7, - Python 3.0
2009-09-24 08:39:05drukkersetmessages: + msg93058
2009-09-24 07:11:12mark.dickinsonsetnosy: + mark.dickinson
messages: + msg93055
2009-09-24 01:03:54r.david.murraysetpriority: low
nosy: + benjamin.peterson

type: behavior
stage: patch review
2009-09-23 20:31:39drukkercreate