classification
Title: py_compile.compile() fails to raise exceptions when writing of target file fails
Type: behavior Stage: committed/rejected
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Arfrever, brett.cannon, eric.snow, ncoghlan, pitrou, python-dev, tim.golden
Priority: normal Keywords:

Created on 2013-02-19 20:47 by Arfrever, last changed 2013-04-24 20:35 by Arfrever. This issue is now closed.

Messages (9)
msg182426 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2013-02-19 20:47
Since d4eb02b6aac9 py_compile.compile() fails to raise exceptions when writing of target file fails.

$ cd /tmp
$ touch test.py
$ mkdir dir
$ chmod a-w dir
mode of ‘dir’ changed from 0755 (rwxr-xr-x) to 0555 (r-xr-xr-x)
$ python3.3 -c 'import py_compile; py_compile.compile("test.py", cfile="/tmp/dir/test.pyc")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.3/py_compile.py", line 141, in compile
    with open(cfile, 'wb') as fc:
PermissionError: [Errno 13] Permission denied: '/tmp/dir/test.pyc'
$ python3.4 -c 'import py_compile; py_compile.compile("test.py", cfile="/tmp/dir/test.pyc")'
$ python3.3 -c 'import py_compile; py_compile.compile("test.py", cfile="/tmp/dir")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.3/py_compile.py", line 141, in compile
    with open(cfile, 'wb') as fc:
IsADirectoryError: [Errno 21] Is a directory: '/tmp/dir'
$ python3.4 -c 'import py_compile; py_compile.compile("test.py", cfile="/tmp/dir")'
$ ls -l dir
total 0
$
msg182432 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-19 21:07
It's because of this try/except to silence failed bytecode creation: http://hg.python.org/cpython/file/d404d33a999c/Lib/importlib/_bootstrap.py#l1105

I hate the py_compile module.
msg183179 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-27 20:00
I figured out what I have to do to make this work properly again to avoid the exception from being swallowed. Roughly:

    # XXX calculate mode (_cache_bytecode)
    # XXX create subdirectories as necessary (set_data)
    # XXX write file (_write_atomic)
    # Above replaces loader._cache_bytecode(file, cfile, bytecode)

That will bypass the try/except block causing the issues.
msg186928 - (view) Author: Roundup Robot (python-dev) Date: 2013-04-14 16:51
New changeset 04aaeae6ee7b by Brett Cannon in branch 'default':
Issue #17244: Don't mask exceptions raised during the creation of
http://hg.python.org/cpython/rev/04aaeae6ee7b
msg187718 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-04-24 18:32
Test broken on Windows:

======================================================================
FAIL: test_exceptions_propagate (test.test_py_compile.PyCompileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_py_compile.py", line 66, in test_exceptions_propagate
    py_compile.compile(self.source_path, self.pyc_path)
AssertionError: OSError not raised

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/1850/steps/test/logs/stdio
msg187726 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-04-24 19:07
Does os.chmod(self.directory, stat.S_IREAD) not work on directories on Windows?
msg187732 - (view) Author: Tim Golden (tim.golden) (Python committer) Date: 2013-04-24 19:28
Essentially: no. The permissions system in Windows is very different 
from that of Unix. The CRT attempts to mimic it, but for things like 
read-onlyness, it does so by setting the (old-style DOS) attributes. 
These are only just meaningful for files, and are meaningless for 
directories. (Or, strictly, have a different meaning since ISTR the 
Explorer shell uses the readonly bit to indicate that a directory has 
some special significance).
msg187736 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-04-24 20:29
Thanks for the info, Tim. I'll add a skip decorator to not run under Windows then.
msg187737 - (view) Author: Roundup Robot (python-dev) Date: 2013-04-24 20:34
New changeset 0803a28dca3c by Brett Cannon in branch 'default':
Issue #17244: Windows doesn't let you set permissions on directories.
http://hg.python.org/cpython/rev/0803a28dca3c
History
Date User Action Args
2013-04-24 20:35:15Arfreversetstage: needs patch -> committed/rejected
2013-04-24 20:34:38brett.cannonsetstatus: open -> closed
resolution: fixed
2013-04-24 20:34:19python-devsetmessages: + msg187737
2013-04-24 20:29:38brett.cannonsetresolution: fixed -> (no value)
messages: + msg187736
stage: committed/rejected -> needs patch
2013-04-24 19:28:22tim.goldensetnosy: + tim.golden
messages: + msg187732
2013-04-24 19:07:16brett.cannonsetmessages: + msg187726
2013-04-24 18:32:13pitrousetstatus: closed -> open

messages: + msg187718
2013-04-14 16:52:25brett.cannonsetstatus: open -> closed
resolution: fixed
stage: test needed -> committed/rejected
2013-04-14 16:51:53python-devsetnosy: + python-dev
messages: + msg186928
2013-02-27 20:00:53brett.cannonsetassignee: brett.cannon
messages: + msg183179
stage: test needed
2013-02-19 21:07:09brett.cannonsetmessages: + msg182432
2013-02-19 20:47:16Arfrevercreate