classification
Title: test___all__ and test_support alter execution environment
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, SilentGhost, ezio.melotti, martin.panter, python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords: needs review, patch

Created on 2015-11-22 08:58 by Arfrever, last changed 2015-11-24 22:44 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
issue25695.diff SilentGhost, 2015-11-22 10:52 review
defer-TESTDIRN.patch martin.panter, 2015-11-23 05:47 review
Messages (6)
msg255088 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2015-11-22 08:58
test___all__ and test_support alter execution environment.
Problem seems to be present only in Python 3.6.

$ python3.5 -m test test___all__ test_support
[1/2] test___all__
[2/2] test_support
All 2 tests OK.
$ python3.6 -m test test___all__ test_support
[1/2] test___all__
Warning -- files was modified by test___all__
[2/2] test_support
Warning -- files was modified by test_support
2 tests altered the execution environment:
    test___all__ test_support


When running these test files separately, problem is only present for test___all__:

$ python3.5 -m test test___all__
[1/1] test___all__
1 test OK.
$ python3.6 -m test test___all__
[1/1] test___all__
Warning -- files was modified by test___all__
1 test altered the execution environment:
    test___all__
$ python3.5 -m test test_support
[1/1] test_support
1 test OK.
$ python3.6 -m test test_support
[1/1] test_support
1 test OK.
msg255093 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2015-11-22 10:52
test_support has this line:

TESTDIRN = os.path.basename(tempfile.mkdtemp(dir='.'))

Which is the culprit. The reason it gets so far as to import the module is due to the "fuzzy logic" check looking for __all__ 
What I'd suggest is replacing simple "in" string search with re.search for '\b__all__\b'. The downside of this is that the test execution time increased from .4 s to .7 s on my machine.
msg255126 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-23 05:47
This was introduced with revision f8fa7bc837a3. I guess I overlooked the new messages because I run the tests with -Wall, which is affected by Issue 18383.

It seems a bad idea to create a temporary directory when you import the test_support module. I propose this patch to defer creation until the one test that uses it is run.

As for your patch Silent Ghost, I’m not sure. It kind of side-steps the real issue by tweaking a heuristic. Maybe we should keep the heuristic as it is, because it is simpler.
msg255139 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-23 08:57
defer-TESTDIRN.patch LGTM.
msg255293 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-11-24 22:17
New changeset fb72d6d7703c by Martin Panter in branch 'default':
Issue #25695: Defer creation of TESTDIRN until the test case is run
https://hg.python.org/cpython/rev/fb72d6d7703c
msg255296 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-24 22:44
This should be fixed now. Thanks for the report Arfrever and the analysis Ghost. I still get these warnings, but they are discussed in Issue 18383:

Warning -- warnings.filters was modified by test___all__
Warning -- warnings.filters was modified by test_warnings
History
Date User Action Args
2015-11-24 22:44:23martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg255296

stage: commit review -> resolved
2015-11-24 22:17:10python-devsetnosy: + python-dev
messages: + msg255293
2015-11-23 08:57:11serhiy.storchakasetnosy: + serhiy.storchaka

messages: + msg255139
stage: patch review -> commit review
2015-11-23 05:47:16martin.pantersetfiles: + defer-TESTDIRN.patch

nosy: + martin.panter
messages: + msg255126

type: behavior
2015-11-22 10:52:10SilentGhostsetfiles: + issue25695.diff

nosy: + SilentGhost
messages: + msg255093

keywords: + needs review, patch
stage: patch review
2015-11-22 08:58:57Arfrevercreate