classification
Title: test directory doubly-nested running tests with -j/--multiprocess
Type: Stage: resolved
Components: Tests Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: chris.jerdonek, flox, pitrou, python-dev, r.david.murray
Priority: normal Keywords: patch

Created on 2012-07-09 01:09 by chris.jerdonek, last changed 2012-07-11 17:26 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
issue-15300-1.patch chris.jerdonek, 2012-07-09 03:11 review
issue-15300-2.patch chris.jerdonek, 2012-07-09 05:06
issue-15300-combined.patch chris.jerdonek, 2012-07-09 05:47 review
issue-15300-3.patch chris.jerdonek, 2012-07-11 03:02 review
Messages (10)
msg165052 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-09 01:09
Running tests using the -j/--multiprocess option doubly-nests the test working directory:

$ ./python.exe -m test -j3 
-->cpython/build/test_python_63955/build/test_python_63956 
$ ./python.exe -m test
-->cpython/build/test_python_63957

It seems like the test directories for different processes should be siblings when running in multiprocessing mode as opposed to doubly-nesting under a new build directory.
msg165055 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-09 03:11
Attaching a small refactoring patch to eliminate some cut-and-paste, prior to fixing this issue.
msg165061 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-09 05:06
And here is the second part.  Combining this with the first patch provides a fix.
msg165066 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-09 05:47
Here is a single combined patch.  I gather this is preferred.
msg165079 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-09 12:11
After this patch, it's clear that TEMPDIR and TESTCWD no longer have to be global variables.

I can make that a separate issue after this one.
msg165185 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-10 12:11
I don't really like the "pre-parsed option" hack. Can't we think of something simpler? For example, why would the child be involved, rather than simply the parent setting the right cwd?

Also, I don't think there's any point in _OPTION_CWD_PARENT_DIR. Just hard-code the literal, it will be more readable.
msg165221 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2012-07-11 03:02
Thanks for your feedback, Antoine, and for causing me to rethink the patch.  This patch is drastically simpler.

Incidentally, I found a reason for this issue in issue 15322.

regrtest calculates the "build" folder relative to sysconfig.get_config_var('srcdir').  Because the latter returns the current working directory instead of the source directory, the build folders were getting nested.
msg165261 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-11 16:07
Thanks, Chris. I haven't tested the patch but it looks fine.
msg165265 - (view) Author: Roundup Robot (python-dev) Date: 2012-07-11 17:23
New changeset 724a6e0e35f0 by Antoine Pitrou in branch '3.2':
Issue #15300: Ensure the temporary test working directories are in the same parent folder when running tests in multiprocess mode from a Python build.
http://hg.python.org/cpython/rev/724a6e0e35f0

New changeset 4752fafb579d by Antoine Pitrou in branch 'default':
Issue #15300: Ensure the temporary test working directories are in the same parent folder when running tests in multiprocess mode from a Python build.
http://hg.python.org/cpython/rev/4752fafb579d
msg165266 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-11 17:26
Committed now.
History
Date User Action Args
2012-07-11 17:26:56pitrousetstatus: open -> closed
versions: + Python 3.2
messages: + msg165266

resolution: fixed
stage: patch review -> resolved
2012-07-11 17:23:27python-devsetnosy: + python-dev
messages: + msg165265
2012-07-11 16:07:38pitrousetmessages: + msg165261
2012-07-11 03:02:21chris.jerdoneksetfiles: + issue-15300-3.patch

messages: + msg165221
2012-07-10 12:11:23pitrousetnosy: + pitrou
messages: + msg165185
2012-07-09 18:48:53pitrousetnosy: + r.david.murray, flox

stage: patch review
2012-07-09 12:11:52chris.jerdoneksetmessages: + msg165079
2012-07-09 05:47:16chris.jerdoneksetfiles: + issue-15300-combined.patch

messages: + msg165066
2012-07-09 05:06:15chris.jerdoneksetfiles: + issue-15300-2.patch

messages: + msg165061
2012-07-09 03:11:32chris.jerdoneksetfiles: + issue-15300-1.patch
keywords: + patch
messages: + msg165055
2012-07-09 01:09:26chris.jerdonekcreate