classification
Title: allow the testsuite to run in the installed location
Type: behavior Stage: needs patch
Components: Tests Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: 17046 17746 17751 17752 17753 17754 17755 17756 17757 17758 17763 17773 21264 21266 22895 25099 25101 25109 Superseder:
Assigned To: doko Nosy List: barry, doko, jibel, ned.deily, pitrou, r.david.murray
Priority: normal Keywords:

Created on 2013-04-16 10:39 by doko, last changed 2019-02-24 22:40 by BreamoreBoy.

Messages (7)
msg187066 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-04-16 10:39
[Meta issue, opening separate issues for test cases]

The default install always installs the testsuite together with all the python batteries. So make sure that you can run the testsuite can run from the installed location too.

 - some tests require the sources to run (using the srcdir macro,
   or files not installed, e.g. xxmodule.c).

 - some tests assume write permissions.

 - some tests assume not being run as root.
msg187090 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-16 15:14
This has been fixed and broken again several times in the past[1].  We need a buildbot that runs the tests installed[2].  Antoine, do you have any thoughts about how to set up such a thing?  I'm willing to host it.

[1] I believe there used to be a program that ran the tests installed, from a cronjob on some dedicated machine somewhere.  It also ran a -R run, I think, as well as being part of a job that built the docs.  When the doc building strategy was changed, somewhere in the process of the transition to Python3 (the program was Python2 only, I believe), it seems to have been taken offline completely.

[2] failing when run as root is a separate problem, and less practical to set a buildbot run for, though since my buildbots run in VMs, it is not completely out of the question.
msg187104 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-04-16 17:28
We could change an existing buildbot, say one of the snakebite buildbots, to add a `make install` and run the tests from it.  I don't think it should be very difficult.  Just make sure ./configure --prefix= has a reasonable value like a dedicated directory (rather than /usr/local), avoid --enable-shared, rm -r on the prefix directories prior to the `make install` step, and cd to some clean directory (other than the source or build directories) then run the tests from `<prefix>/bin/python -m test (or test.regrtest for 2.7) ...
msg187106 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-16 17:37
It occurs to me that the tricky bit is that the install directory contents should be (a) read-only (easy) and ideally (b) owned by root (a little trickier, but doable with a setuid script or tailored sudo command).
msg187124 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2013-04-16 22:27
it doesn't have to be root, just another user as for running the testsuite.
msg187125 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-16 22:30
Right, it doesn't *have* to be root, but that would be the typical situation when the tests are run with Python installed.  Having it be any other user still requires some privilege increase.
msg223253 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-16 18:58
How do we stand with eight outstanding issues here?
History
Date User Action Args
2019-02-24 22:40:16BreamoreBoysetnosy: - BreamoreBoy
2015-09-14 12:18:55serhiy.storchakasetdependencies: + test_compileall fails when run by unprivileged user on installed Python, test_zipfile failure when run by unprivileged user with installed Python
2015-09-14 11:27:30dokosetdependencies: + test_code_module tests fail when run from the installed location
2014-11-18 17:11:10dokosetdependencies: + test failure introduced by the fix for issue #22462
2014-07-16 18:58:53BreamoreBoysetnosy: + BreamoreBoy

messages: + msg223253
versions: + Python 3.5, - Python 3.3
2014-04-16 19:12:57dokosetdependencies: + test_zipfile fails to run in the installed location
2014-04-16 19:00:12dokosetdependencies: + test_compileall fails to build in the installed location
2013-04-17 10:39:17dokosetdependencies: + test_pydoc fails with the installed testsuite (2.7)
2013-04-16 22:30:09r.david.murraysetmessages: + msg187125
2013-04-16 22:27:26dokosetmessages: + msg187124
2013-04-16 22:23:21dokosetdependencies: + test_subprocess test_executable_without_cwd fails when run with installed python
2013-04-16 17:37:27r.david.murraysetmessages: + msg187106
2013-04-16 17:28:24ned.deilysetnosy: + ned.deily
messages: + msg187104
2013-04-16 15:23:54dokosetdependencies: + test_pydoc fails with the installed testsuite
2013-04-16 15:14:24r.david.murraysetnosy: + r.david.murray, pitrou
type: enhancement -> behavior
messages: + msg187090
2013-04-16 13:52:22dokosetdependencies: + test_site fails when the user does not have a home directory
2013-04-16 13:08:29barrysetnosy: + barry
2013-04-16 11:39:38dokosetdependencies: + test_executable_without_cwd fails when run in the installed location
2013-04-16 11:32:57dokosetdependencies: + test_syntax_error fails when run in the installed location
2013-04-16 11:23:49dokosetdependencies: + test_builtin assumes LANG=C
2013-04-16 11:14:49dokosetdependencies: + test_ctypes assumes LANG=C LC_ALL=C
2013-04-16 10:54:13dokosetdependencies: + test_zipfile: requires write access to test and email.test
2013-04-16 10:49:54dokosetdependencies: + many distutils tests fail when run from the installed location
2013-04-16 10:47:23dokosetdependencies: + ctypes/test/test_macholib.py fails when run from the installed location
2013-04-16 10:40:20dokosetnosy: + jibel
2013-04-16 10:39:57dokosetdependencies: + test_shutil.TestWhich.test_non_matching_mode fails when running as root
2013-04-16 10:39:40dokocreate