classification
Title: [2.7] test_regrtest: test_env_changed() fails on x86 Windows XP VS9.0 2.7
Type: Stage: resolved
Components: Tests, Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: pablogsal, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2018-07-02 10:05 by vstinner, last changed 2018-11-29 22:38 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10804 merged vstinner, 2018-11-29 22:09
Messages (5)
msg320865 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-02 10:05
http://buildbot.python.org/all/#/builders/105/builds/182

d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build>"d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\PC\VS9.0\\python_d"  -Wd -3 -E -tt "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\PC\VS9.0\\..\..\Lib\test\regrtest.py" -uall -rwW --slowest -j2      
== CPython 2.7.15+ (default, Jun 30 2018, 07:15:20) [MSC v.1500 32 bit (Intel)]
==   Windows-XP-5.1.2600-SP3 little-endian
==   d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_3044
== CPU count: 1
Using random seed 5527885
Run tests in parallel using 2 child processes
...
0:11:15 [256/403/1] test_regrtest failed
d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\support\__init__.py:808: RuntimeWarning: tests may fail, unable to create temp dir: d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_2228
  with temp_dir(path=name, quiet=quiet) as temp_path:
test test_regrtest failed -- Traceback (most recent call last):
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 662, in test_env_changed
    self.check_executed_tests(output, [testname], env_changed=testname)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 140, in check_executed_tests
    self.check_line(output, regex)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 87, in check_line
    self.assertRegexpMatches(output, regex)
AssertionError: Regexp didn't match: '^1 test altered the execution environment:\\n    test_regrtest_noop33$' not found in 'Run tests sequentially\n0:00:00 [1/1] test_regrtest_noop33\n\n== Tests result: SUCCESS ==\n\n1 test OK.\n\nTotal duration: 131 ms\nTests result: SUCCESS\n'
...
Re-running failed tests in verbose mode
Re-running test 'test_regrtest' in verbose mode
d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\support\__init__.py:808: RuntimeWarning: tests may fail, unable to create temp dir: d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_2336
  with temp_dir(path=name, quiet=quiet) as temp_path:
test_module_autotest (test.test_regrtest.ProgramsTestCase) ... ok
test_module_from_test_autotest (test.test_regrtest.ProgramsTestCase) ... ok
test_module_regrtest (test.test_regrtest.ProgramsTestCase) ... ok
test_module_test (test.test_regrtest.ProgramsTestCase) ... ok
test_pcbuild_rt (test.test_regrtest.ProgramsTestCase) ... skipped 'Tools/buildbot/test.bat requires PCbuild build, found PC'
test_script_autotest (test.test_regrtest.ProgramsTestCase) ... ok
test_script_regrtest (test.test_regrtest.ProgramsTestCase) ... ok
test_tools_buildbot_test (test.test_regrtest.ProgramsTestCase) ... skipped 'Tools/buildbot/test.bat requires PCbuild build, found PC'
test_coverage (test.test_regrtest.ArgsTestCase) ... ok
test_crashed (test.test_regrtest.ArgsTestCase) ... ok
test_env_changed (test.test_regrtest.ArgsTestCase) ... FAIL
test_failing_test (test.test_regrtest.ArgsTestCase) ... ok
test_forever (test.test_regrtest.ArgsTestCase) ... ok
test_fromfile (test.test_regrtest.ArgsTestCase) ... ok
test_huntrleaks (test.test_regrtest.ArgsTestCase) ... ok
test_huntrleaks_fd_leak (test.test_regrtest.ArgsTestCase) ... ok
test_interrupted (test.test_regrtest.ArgsTestCase) ... ok
test_list_cases (test.test_regrtest.ArgsTestCase) ... ok
test_list_tests (test.test_regrtest.ArgsTestCase) ... ok
test_matchfile (test.test_regrtest.ArgsTestCase) ... ok
test_random (test.test_regrtest.ArgsTestCase) ... ok
test_rerun_fail (test.test_regrtest.ArgsTestCase) ... ok
test_resources (test.test_regrtest.ArgsTestCase) ... ok
test_slow_interrupted (test.test_regrtest.ArgsTestCase) ... ok
test_slowest (test.test_regrtest.ArgsTestCase) ... ok
test_format_duration (test.test_regrtest.TestUtils) ... ok

======================================================================
FAIL: test_env_changed (test.test_regrtest.ArgsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 665, in test_env_changed
    output = self.run_tests("--fail-env-changed", testname, exitcode=3)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 335, in run_tests
    return self.run_python(cmdargs, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 216, in run_python
    proc = self.run_command(args, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 211, in run_command
    self.fail(msg)
AssertionError: Command ['d:\\cygwin\\home\\db3l\\buildarea\\2.7.bolen-windowsvs9\\build\\PC\\VS9.0\\python_d.exe', '-m', 'test', '--testdir=c:\\docume~1\\db3l\\locals~1\\temp\\tmpvaq672', '--fail-env-changed', 'test_regrtest_noop33'] failed with exit code 0

stdout:
---
Run tests sequentially
0:00:00 [1/1] test_regrtest_noop33

== Tests result: SUCCESS ==

1 test OK.

Total duration: 10 ms
Tests result: SUCCESS

---

stderr:
---
d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\support\__init__.py:808: RuntimeWarning: tests may fail, unable to create temp dir: d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_3992
  with temp_dir(path=name, quiet=quiet) as temp_path:
---


----------------------------------------------------------------------
Ran 26 tests in 16.384s

FAILED (failures=1, skipped=2)
1 test failed again:
    test_regrtest

== Tests result: FAILURE then FAILURE ==

359 tests OK.

10 slowest tests:
- test_mmap: 357.3s
- test_largefile: 277.8s
- test_io: 275.1s
- test_multiprocessing: 194.6s
- test_decimal: 57.8s
- test_lib2to3: 45.3s
- test_tokenize: 42.2s
- test_subprocess: 36.7s
- test_file2k: 36.7s
- test_ssl: 30.9s

1 test failed:
    test_regrtest

43 tests skipped:
    test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
    test_cd test_cl test_commands test_crypt test_curses test_dbm
    test_dl test_epoll test_fcntl test_fork1 test_gdb test_gdbm
    test_gl test_grp test_imgfile test_ioctl test_kqueue
    test_linuxaudiodev test_macos test_macostools test_mhlib test_nis
    test_openpty test_ossaudiodev test_pipes test_poll test_posix
    test_pty test_pwd test_readline test_resource test_scriptpackages
    test_spwd test_sunaudiodev test_threadsignals test_wait3
    test_wait4 test_zipfile64
2 skips unexpected on win32:
    test_gdb test_readline

1 re-run test:
    test_regrtest

Total duration: 24 min 6 sec
Tests result: FAILURE then FAILURE
test test_regrtest failed -- Traceback (most recent call last):
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 665, in test_env_changed
    output = self.run_tests("--fail-env-changed", testname, exitcode=3)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 335, in run_tests
    return self.run_python(cmdargs, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 216, in run_python
    proc = self.run_command(args, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 211, in run_command
    self.fail(msg)
AssertionError: Command ['d:\\cygwin\\home\\db3l\\buildarea\\2.7.bolen-windowsvs9\\build\\PC\\VS9.0\\python_d.exe', '-m', 'test', '--testdir=c:\\docume~1\\db3l\\locals~1\\temp\\tmpvaq672', '--fail-env-changed', 'test_regrtest_noop33'] failed with exit code 0

stdout:
---
Run tests sequentially
0:00:00 [1/1] test_regrtest_noop33

== Tests result: SUCCESS ==

1 test OK.

Total duration: 10 ms
Tests result: SUCCESS

---

stderr:
---
d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\support\__init__.py:808: RuntimeWarning: tests may fail, unable to create temp dir: d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_3992
  with temp_dir(path=name, quiet=quiet) as temp_path:
---


program finished with exit code 2
elapsedTime=1447.752000
msg322161 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-22 19:17
It seems that this buildbot failure is related to this issue:

https://buildbot.python.org/all/#/builders/105/builds/192


1 test failed:
    test_regrtest
43 tests skipped:
    test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
    test_cd test_cl test_commands test_crypt test_curses test_dbm
    test_dl test_epoll test_fcntl test_fork1 test_gdb test_gdbm
    test_gl test_grp test_imgfile test_ioctl test_kqueue
    test_linuxaudiodev test_macos test_macostools test_mhlib test_nis
    test_openpty test_ossaudiodev test_pipes test_poll test_posix
    test_pty test_pwd test_readline test_resource test_scriptpackages
    test_spwd test_sunaudiodev test_threadsignals test_wait3
    test_wait4 test_zipfile64
2 skips unexpected on win32:
    test_gdb test_readline
1 re-run test:
    test_regrtest
Total duration: 29 min 51 sec
Tests result: FAILURE then FAILURE
test test_regrtest failed -- Traceback (most recent call last):
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 662, in test_env_changed
    self.check_executed_tests(output, [testname], env_changed=testname)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 140, in check_executed_tests
    self.check_line(output, regex)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 87, in check_line
    self.assertRegexpMatches(output, regex)
AssertionError: Regexp didn't match: '^1 test altered the execution environment:\\n    test_regrtest_noop33$' not found in 'Run tests sequentially\n0:00:00 [1/1] test_regrtest_noop33\n\n== Tests result: SUCCESS ==\n\n1 test OK.\n\nTotal duration: 20 ms\nTests result: SUCCESS\n'


Both failures mention "test_regrtest_noop33".
msg322260 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-23 21:27
Another failure in sx86 Windows XP VS9.0 2.7:

https://buildbot.python.org/all/#/builders/105/builds/195

Total duration: 28 min 12 sec
Tests result: FAILURE then FAILURE
test test_regrtest failed -- Traceback (most recent call last):
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 665, in test_env_changed
    output = self.run_tests("--fail-env-changed", testname, exitcode=3)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 335, in run_tests
    return self.run_python(cmdargs, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 216, in run_python
    proc = self.run_command(args, **kw)
  File "d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\test_regrtest.py", line 211, in run_command
    self.fail(msg)
AssertionError: Command ['d:\\cygwin\\home\\db3l\\buildarea\\2.7.bolen-windowsvs9\\build\\PC\\VS9.0\\python_d.exe', '-m', 'test', '--testdir=c:\\docume~1\\db3l\\locals~1\\temp\\tmpqsasla', '--fail-env-changed', 'test_regrtest_noop33'] failed with exit code 0
stdout:
---
Run tests sequentially
0:00:00 [1/1] test_regrtest_noop33
== Tests result: SUCCESS ==
1 test OK.
Total duration: 20 ms
Tests result: SUCCESS
---
stderr:
---
d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\lib\test\support\__init__.py:808: RuntimeWarning: tests may fail, unable to create temp dir: d:\cygwin\home\db3l\buildarea\2.7.bolen-windowsvs9\build\build\test_python_2916
  with temp_dir(path=name, quiet=quiet) as temp_path:
---
msg330732 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-29 22:11
Five months later nobody has any kind of clue explaining why the test randomly fails on this specific buildbot, whereas it's reliable on all other buildbots and all Python 3.x versions. So I wrote PR 10804 to skip the test on Windows. The feature will remain to be tested on all other platforms, and it's still tested on Windows on Python 3.x.
msg330734 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-29 22:38
New changeset f681e93e46925e862a8b0fa4b6e9e341fd8de3c2 by Victor Stinner in branch '2.7':
bpo-34021: Windows skips test_regrtest.test_env_changed() (GH-10804)
https://github.com/python/cpython/commit/f681e93e46925e862a8b0fa4b6e9e341fd8de3c2
History
Date User Action Args
2018-11-29 22:38:47vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-11-29 22:38:37vstinnersetmessages: + msg330734
2018-11-29 22:11:57vstinnersetmessages: + msg330732
2018-11-29 22:09:19vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request10051
2018-07-23 21:27:19pablogsalsetmessages: + msg322260
2018-07-22 19:17:48pablogsalsetnosy: + pablogsal
messages: + msg322161
2018-07-02 10:05:51vstinnersetcomponents: + Tests
2018-07-02 10:05:46vstinnersettitle: test_regrtest: test_env_changed() fails on x86 Windows XP VS9.0 2.7 -> [2.7] test_regrtest: test_env_changed() fails on x86 Windows XP VS9.0 2.7
2018-07-02 10:05:38vstinnercreate