classification
Title: test_compile_path fails when python has been installed
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Rosuav, berker.peksag, brett.cannon, christian.heimes, python-dev, vstinner, xdegaye, yan12125
Priority: low Keywords: patch

Created on 2016-06-15 07:20 by xdegaye, last changed 2016-09-28 17:35 by Rosuav. This issue is now closed.

Files
File name Uploaded Description Edit
compile-nearly-all.patch Rosuav, 2016-09-07 07:00 review
issue27322_v2.diff berker.peksag, 2016-09-13 05:59 review
issue27322_v3.diff berker.peksag, 2016-09-27 23:19 review
Messages (17)
msg268607 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-06-15 07:20
======================================================================
FAIL: test_compile_path (test.test_compileall.CompileallTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xavier/src/python/default/Lib/test/test_compileall.py", line 118, in test_compile_path
    self.assertTrue(compileall.compile_path(quiet=2))
AssertionError: False is not true


Changing the 'quiet' flag value to 0, gives:

test_compile_path (test.test_compileall.CompileallTests) ... Skipping current directory
Listing '/usr/local/lib/python36.zip'...
Can't list '/usr/local/lib/python36.zip'
Listing '/home/xavier/src/python/default/Lib'...
Listing '/home/xavier/src/python/default/Lib/plat-x86_64-linux-gnu'...
Can't list '/home/xavier/src/python/default/Lib/plat-x86_64-linux-gnu'
Listing '/home/xavier/src/python/default/build/lib.linux-x86_64-3.6-pydebug'...
Listing '/home/xavier/.local/lib/python3.6/site-packages'...
Listing '/usr/local/lib/python3.6/site-packages'...
Compiling '/usr/local/lib/python3.6/site-packages/easy_install.py'...
*** PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/site-packages/__pycache
__/easy_install.cpython-36.pyc.139734863394416'
FAIL


Note that this test has already been fixed for another problem in issue 26101.
msg274771 - (view) Author: Chris Angelico (Rosuav) * Date: 2016-09-07 07:00
Just ran into the same issue, but in a slightly different way, and with slightly different consequences. You can actually eliminate this failure by manually running compileall on the affected directory:

sudo python3 -m compileall /usr/local/lib/python3.6/site-packages

In other words, a test failure can come and go depending on the presence of other files on the system.

IMO the test should (a) restrict itself to directories within the build tree, and (b) call compile_path with force=True. That should make it more consistent. Something like the attached patch.
msg276034 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-09-12 13:03
Or we can just skip the test like the other tests with self._skip_if_sys_path_not_writable()
msg276035 - (view) Author: Chris Angelico (Rosuav) * Date: 2016-09-12 13:07
Huh. Yeah, that would work really nicely. Thanks Christian!

Does this count as a bugfix? I think we've missed 3.6 feature freeze.
msg276036 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-12 13:08
New changeset e9e4d1b5b4b5 by Christian Heimes in branch 'default':
Issue #27322: skip test_compile_path when sys.path is not writeable.
https://hg.python.org/cpython/rev/e9e4d1b5b4b5
msg276037 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-09-12 13:11
AFAIK Ned hasn't tagged the beta yet. Let's see if this fixes the failing Gentoo build bot.
msg276041 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-12 13:22
New changeset f7d198fc4c96 by Christian Heimes in branch 'default':
Issue #27322: back out the commit. needs to be addressed after beta1.
https://hg.python.org/cpython/rev/f7d198fc4c96
msg276181 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-13 05:59
Here is a patch that reuses the helper in CommandLineTests.
msg276207 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-13 07:55
test_compileall depends on sys.path? It doesn't seem reliable to me.

I suggest to create a temporary directory and replace sys.path to only have one item: this directory.

I don't know which files are expected to be there, but we can probably copy .py to this temporary directory.

What do you think?
msg276312 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-09-13 18:04
That seems reasonable to me. There is a context manager in test.test_importlib.util which can help with changing sys.path temporarily.
msg277496 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-27 07:01
Any update on the issue?

http://buildbot.python.org/all/builders/x86%20Gentoo%20Installed%20with%20X%203.x is still failing.
msg277499 - (view) Author: Chris Angelico (Rosuav) * Date: 2016-09-27 07:26
I'm not sure. The patch got backed out with little explanation, so presumably it broke something somewhere, but I don't know what.
msg277563 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-27 23:19
Here's an updated patch.
msg277632 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-28 14:37
New changeset e53984b87cf0 by Berker Peksag in branch '3.6':
Issue #27322: Set sys.path to a temp dir in test_compile_path
https://hg.python.org/cpython/rev/e53984b87cf0

New changeset a78446a65b1d by Berker Peksag in branch 'default':
Issue #27322: Merge from 3.6
https://hg.python.org/cpython/rev/a78446a65b1d
msg277647 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-09-28 16:57
The buildbot is now green. Closing this as fixed.
msg277648 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-28 17:12
Thanks for your fix :-)
msg277649 - (view) Author: Chris Angelico (Rosuav) * Date: 2016-09-28 17:35
Also working for me. Thanks!
History
Date User Action Args
2016-09-28 17:35:21Rosuavsetmessages: + msg277649
2016-09-28 17:12:26vstinnersetmessages: + msg277648
2016-09-28 16:57:52berker.peksagsetstatus: open -> closed
resolution: fixed
messages: + msg277647

stage: patch review -> resolved
2016-09-28 14:45:31berker.peksagsetversions: - Python 3.5
2016-09-28 14:37:26python-devsetmessages: + msg277632
2016-09-27 23:19:14berker.peksagsetfiles: + issue27322_v3.diff

messages: + msg277563
2016-09-27 07:26:43Rosuavsetmessages: + msg277499
2016-09-27 07:01:39vstinnersetmessages: + msg277496
2016-09-13 18:04:50brett.cannonsetmessages: + msg276312
2016-09-13 07:55:03vstinnersetmessages: + msg276207
2016-09-13 05:59:08berker.peksagsetfiles: + issue27322_v2.diff
versions: + Python 3.5, Python 3.7
nosy: + berker.peksag

messages: + msg276181

stage: needs patch -> patch review
2016-09-12 16:26:34yan12125setnosy: + yan12125
2016-09-12 13:22:40python-devsetmessages: + msg276041
2016-09-12 13:11:06christian.heimessetmessages: + msg276037
2016-09-12 13:08:54python-devsetnosy: + python-dev
messages: + msg276036
2016-09-12 13:07:15Rosuavsetmessages: + msg276035
2016-09-12 13:03:33christian.heimessetnosy: + christian.heimes
messages: + msg276034
2016-09-07 07:00:19Rosuavsetfiles: + compile-nearly-all.patch

nosy: + Rosuav
messages: + msg274771

keywords: + patch
2016-06-15 07:20:19xdegayecreate