This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [PEP 3147] compileall.compile_file() creates empty __pycache__ directories for non-.py files
Type: Stage:
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: barry Nosy List: Arfrever, barry
Priority: normal Keywords: patch

Created on 2010-04-28 20:01 by Arfrever, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
compileall.patch Arfrever, 2010-04-28 20:01 Patch for compileall.compile_file()
8563.patch barry, 2010-04-29 15:49
Messages (10)
msg104458 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-04-28 20:01
compileall.compile_file() creates empty __pycache__ directories for non-.py files.
This problem usually occurs when compileall.compile_file() is called by compileall.compile_dir() and a subdirectory contains non-code files (e.g. locales, images, templates, documentation).
__pycache__ directories also should not be created when generation of .pyc / .pyo files failed due to e.g. SyntaxErrors.
I'm attaching the patch.

$ mkdir test
$ touch test/file
$ tree test
test
└── file

0 directories, 1 file
$ python3.2 -c 'import compileall; compileall.compile_file("test/file")'
$ tree test
test
├── file
└── __pycache__

1 directory, 1 file
msg104528 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-04-29 15:49
Thanks for finding this bug.  I'm not sure about the second chunk of your diff (the one that removes __pycache__).  In the future, it would help to include a unittest case that illustrates the problem.  Here's a patch that includes a test for that specific problem, along with a narrowed down patch.  Can you explain why the other chunk is necessary (preferably with a unittest)?
msg104533 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-04-29 15:57
Here's an example, which shows that an empty __pycache__ has been created:

$ mkdir test
$ echo ":" > test/file.py
$ tree test
test
└── file.py

0 directories, 1 file
$ python3.2 -c 'import compileall; compileall.compile_file("test/file.py")'
Compiling test/file.py ...
***   File "test/file.py", line 1
    :
    ^
SyntaxError: invalid syntax

$ tree test
test
├── file.py
└── __pycache__

1 directory, 1 file
msg104552 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-04-29 18:08
Hmm. I'm not sure that getting a SyntaxError is grounds for removing the __pycache__ directory.  I think I will commit the fix without that change.
msg104556 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-04-29 18:16
__pycache__ directory could be removed only when os.mkdir(cache_dir) has succeeded. Empty __pycache__ directory existing before call to compileall.compile_file() wouldn't be removed.
msg104563 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-04-29 18:41
I guess I don't see the point in removing __pycache__ when the only .py file in the package directory has a SyntaxError.  I think that will be a rare case. Won't there usually be other .py files that compile just fine?
msg104564 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-04-29 18:43
r80626
msg104566 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-04-29 18:47
compileall might be used to check if given package supports Python 3. There can be many .py files and all of them can contain syntax specific to Python 2.
msg104567 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2010-04-29 18:52
Hmm.  In that case, I think you'd be expecting possible errors and would either have to clean up __pycache__ yourself or turn on legacy mode.
msg104569 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2010-04-29 19:02
I think that compilation of a single file should try to be an atomic operation, which doesn't leave an empty directory after failure.
History
Date User Action Args
2022-04-11 14:57:00adminsetgithub: 52809
2010-04-29 19:02:17Arfreversetmessages: + msg104569
2010-04-29 18:52:18barrysetmessages: + msg104567
2010-04-29 18:47:12Arfreversetmessages: + msg104566
2010-04-29 18:43:23barrysetstatus: open -> closed
resolution: fixed
messages: + msg104564
2010-04-29 18:41:17barrysetmessages: + msg104563
2010-04-29 18:16:37Arfreversetmessages: + msg104556
2010-04-29 18:08:49barrysetmessages: + msg104552
2010-04-29 15:57:40Arfreversetmessages: + msg104533
2010-04-29 15:50:02barrysetassignee: barry
2010-04-29 15:49:52barrysetfiles: + 8563.patch

messages: + msg104528
2010-04-28 20:01:40Arfrevercreate