classification
Title: test_compileall fails randomly on Windows when tests are run in parallel
Type: Stage:
Components: Tests Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jkloth, pablogsal, vstinner
Priority: normal Keywords:

Created on 2019-06-24 13:40 by vstinner, last changed 2020-01-06 22:20 by vstinner.

Messages (2)
msg346402 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-24 13:40
I'm not sure that it's a good practice to rewrite .pyc files of the stdlib while tests are run in parallel.

Maybe the test should copy all .py files of the stdlib into a temporary directory and work there, to not impact other tests run in parallel.

Example on AMD64 Windows7 SP1 3.8:
https://buildbot.python.org/all/#/builders/208/builds/105

======================================================================
FAIL: test_no_args_respects_force_flag (test.test_compileall.CommmandLineTestsWithSourceEpoch)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_py_compile.py", line 30, in wrapper
    return fxn(*args, **kwargs)
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_py_compile.py", line 20, in wrapper
    return fxn(*args, **kwargs)
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_compileall.py", line 322, in test_no_args_respects_force_flag
    self.assertRunOK('-f', PYTHONPATH=self.directory)
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\test_compileall.py", line 271, in assertRunOK
    rc, out, err = script_helper.assert_python_ok(
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\support\script_helper.py", line 157, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\support\script_helper.py", line 143, in _assert_python
    res.fail(cmd_line)
  File "C:\buildbot.python.org\3.8.kloth-win64\build\lib\test\support\script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['C:\\buildbot.python.org\\3.8.kloth-win64\\build\\PCbuild\\amd64\\python_d.exe', '-X', 'faulthandler', '-S', '-m', 'compileall', '-f']

stdout:
---
(... truncated stdout ...)ling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\ipaddress.py'...
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\keyword.py'...
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\linecache.py'...
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\locale.py'...
(...)
*** PermissionError: [WinError 5] Access is denied: 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\__pycache__\\site.cpython-38.pyc.25436544' -> 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\__pycache__\\site.cpython-38.pyc'
(...)
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\zipapp.py'...
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\zipfile.py'...
Compiling 'C:\\buildbot.python.org\\3.8.kloth-win64\\build\\lib\\zipimport.py'...
---

stderr:
---

---

----------------------------------------------------------------------

Ran 81 tests in 25.827s

FAILED (failures=1, skipped=2)
test test_compileall failed
msg359474 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-06 22:20
x86 Windows7 3.8:
https://buildbot.python.org/all/#/builders/223/builds/59

FAIL: test_no_args_respects_force_flag (test.test_compileall.CommmandLineTestsNoSourceEpoch)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\test_py_compile.py", line 20, in wrapper
    return fxn(*args, **kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\test_py_compile.py", line 20, in wrapper
    return fxn(*args, **kwargs)
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\test_compileall.py", line 322, in test_no_args_respects_force_flag
    self.assertRunOK('-f', PYTHONPATH=self.directory)
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\test_compileall.py", line 271, in assertRunOK
    rc, out, err = script_helper.assert_python_ok(
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\support\script_helper.py", line 157, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\support\script_helper.py", line 143, in _assert_python
    res.fail(cmd_line)
  File "D:\cygwin\home\db3l\buildarea\3.8.bolen-windows7\build\lib\test\support\script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['D:\\cygwin\\home\\db3l\\buildarea\\3.8.bolen-windows7\\build\\PCbuild\\win32\\python_d.exe', '-X', 'faulthandler', '-S', '-m', 'compileall', '-f']

stdout:
---
(...)

Compiling 'D:\\cygwin\\home\\db3l\\buildarea\\3.8.bolen-windows7\\build\\lib\\socketserver.py'...

Compiling 'D:\\cygwin\\home\\db3l\\buildarea\\3.8.bolen-windows7\\build\\lib\\sre_compile.py'...

*** PermissionError: [WinError 5] Access is denied: 'D:\\cygwin\\home\\db3l\\buildarea\\3.8.bolen-windows7\\build\\lib\\__pycache__\\sre_compile.cpython-38.pyc.12753880' -> 'D:\\cygwin\\home\\db3l\\buildarea\\3.8.bolen-windows7\\build\\lib\\__pycache__\\sre_compile.cpython-38.pyc'
History
Date User Action Args
2020-01-06 22:20:27vstinnersetmessages: + msg359474
2019-06-24 16:14:43jklothsetnosy: + jkloth
2019-06-24 13:40:26vstinnercreate