classification
Title: [Windows] support.rmtree() should retry on WindowsError: [Error 32] The process cannot access the file because it is being used by another process
Type: Stage: resolved
Components: Tests, Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jkloth, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2017-05-10 15:50 by vstinner, last changed 2017-05-11 00:42 by vstinner. This issue is now closed.

Messages (5)
msg293428 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-10 15:50
test_startfile uses a weak time.sleep(0.1) to wait until the process started by os.startfile() completes. Instead of a sleep, can't support.rmtree() retry on "WindowsError: [Error 32] The process cannot access the file because it is being used by another process" error? For example, retry every 100 ms during 10 seconds?

It's easy to reproduce the bug on Python 2.7: just remove the time.sleep() call in tests.

http://buildbot.python.org/all/builders/x86%20Windows7%202.7/builds/145/steps/test/logs/stdio

0:23:42 [363/402/1] test_startfile crashed

[1, 5.638000011444092]
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\regrtest.py", line 1877, in <module>
    main_in_temp_cwd()
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\regrtest.py", line 1864, in main_in_temp_cwd
    main()
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 769, in temp_cwd
    rmtree(name)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 293, in rmtree
    _rmtree(path)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 249, in _rmtree
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 202, in _waitfor
    func(pathname)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 249, in <lambda>
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
  File "D:\cygwin\home\db3l\buildarea\2.7.bolen-windows7\build\lib\test\support\__init__.py", line 197, in _force_run
    return func(*args)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'D:\\cygwin\\home\\db3l\\buildarea\\2.7.bolen-windows7\\build\\build\\test_python_2944'

--

If it's too complex to implement a retry (too complex? really?), an obvious hack is to increase the sleep in test_startfile.py from 100 ms to 200 ms...
msg293429 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-10 15:52
For a very generic "retry" library, see https://pypi.python.org/pypi/tenacity

But I don't think that we need such advanced library, a simple loop should do the trick ;-)
msg293432 - (view) Author: Jeremy Kloth (jkloth) * Date: 2017-05-10 16:21
See http://bugs.python.org/issue15526
msg293433 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-10 16:33
> See http://bugs.python.org/issue15526

Oh, thanks for the hint :-)

I proposed a "backport" (modified for 2.7): https://github.com/python/cpython/pull/1537
msg293467 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-11 00:42
My following backport fixed test_startfile on x86 Windows7 3.x, so I close the issue.

New changeset 3837d9797c14c13d170256143c841d29645e772a by Victor Stinner in branch '2.7':
bpo-15526: test_startfile changes the cwd (#1537)
https://github.com/python/cpython/commit/3837d9797c14c13d170256143c841d29645e772a
History
Date User Action Args
2017-05-11 00:42:32vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg293467

stage: resolved
2017-05-10 16:33:21vstinnersetmessages: + msg293433
2017-05-10 16:21:28jklothsetmessages: + msg293432
2017-05-10 15:52:50vstinnersetmessages: + msg293429
2017-05-10 15:50:11vstinnercreate