This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Sometimes buildbot fails to cleanup working copy
Type: behavior Stage: resolved
Components: Tests, Windows Versions: Python 3.5
Status: closed Resolution: fixed
Dependencies: Superseder: Update Windows build batch scripts
View: 21907
Assigned To: zach.ware Nosy List: BreamoreBoy, db3l, ocean-city, pitrou, steve.dower, zach.ware
Priority: normal Keywords: buildbot, patch

Created on 2010-09-28 16:34 by ocean-city, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit
py3k_buildbot_error_in_clean_faze_v2.patch ocean-city, 2010-12-07 08:49 review
Messages (10)
msg117536 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2010-09-28 16:34
Sometimes, buildbot fails to clean working copy.


Clean started: Project: _hashlib, Configuration: Debug|Win32
Deleting intermediate and output files for project '_hashlib', configuration 'Debug|Win32'
_hashlib : error PRJ0008 : Could not delete file 'c:\buildslave\2.7.moore-windows\build\PCbuild\_hashlib_d.pyd'.
Make sure that the file is not open by another process and is not write-protected.
_hashlib - 1 error(s), 0 warning(s)

Build complete: 8 Projects succeeded, 17 Projects failed, 1 Projects skipped


This might cause checkout error next time. Maybe attached
patch can improve this situation. Thanks.
msg123562 - (view) Author: David Bolen (db3l) * Date: 2010-12-07 16:17
I wonder if it might be better to have kill_python run at the tail end of test.bat, since there's a potential window where losing a connection to the build master might not run the clean step, so still risk blocking a future build after a reconnect due to still running processes.
msg127552 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-30 21:15
As far as I can tell, the same change also needs to be done in build-amd64.bat and clean-amd64.bat.
msg127553 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-30 21:19
By the way, the order of these lines in clean-amd64.bat looks wrong:

cd PCbuild
@echo Deleting .pyc/.pyo files ...
del /s Lib\*.pyc Lib\*.pyo
msg127554 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-30 21:23
Oh, and the regrtest command line in test-amd64.bat lacks some options ("-rwW -n") compared to test.bat.
msg127555 - (view) Author: David Bolen (db3l) * Date: 2011-01-30 21:29
Probably - it's just a general sequencing change, so I suppose should apply equally to all platforms.  I suppose even better would be to consolidate the two clean scripts into one (with a parameter for 32 v. 64), but just patching both is less of a change.  Unrelated, but it's interesting how the amd64 script makes sure to be in the PCBuild tree before deleting byte compiled files, but the standard script doesn't.

Oh, I should probably also update my prior thought in this ticket, since based on a python-dev thread Hirokazu pointed out that just having things at the end of test.bat could be exposed if the actual test itself never returns to the script.

That might have higher odds than the build master losing contact with the slave between build and clean step, so the original patch is likely the better choice.  Or maybe do it in both both places.  

But just moving things to clean first would cover the vast majority of cases I seem to see on my build slaves, which only rarely lose contact with the master.
msg221689 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-27 14:39
@Steve I'm assuming that this is covered by the work you're doing for 3.5 builds, am I correct?
msg221691 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2014-06-27 15:06
Probably, but that work is not going to be checked in for a while (until we have guarantees that we'll be able to use VC14 and there's a 'go-live' version available). If this is causing problems now, it should be fixed.

The patch looks fine to me, but Zachary Ware does more of the buildbot script maintenance (not sure if I can nosy him without being able to log in...)

Sent from my Windows Phone
From: Mark Lawrence<>
Sent: ‎6/‎27/‎2014 9:39
To: Steve Dower<>
Subject: [issue9973] Sometimes buildbot fails to cleanup working copy

Mark Lawrence added the comment:

@Steve I'm assuming that this is covered by the work you're doing for 3.5 builds, am I correct?

components: +Windows
nosy: +BreamoreBoy, steve.dower
versions: +Python 3.4, Python 3.5 -Python 3.1, Python 3.2

Python tracker <>
msg221692 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-06-27 16:51
The real simple method here would be to replace clean[-amd64].bat with a call to kill_python_d (if it exists), followed by an "hg --config extensions.purge= purge --all".  That ought to give as much of a guarantee of a clean slate as possible, with the added benefit of hg purge reporting files that it just can't remove.
msg222498 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-07-07 18:54
This should be fixed by #21907 for 3.5, at least; other versions can either have #21907 backported (in whole or in part), or continue to live with it (like they have for the past 4 years).
Date User Action Args
2022-04-11 14:57:07adminsetgithub: 54182
2014-07-07 18:54:23zach.waresetstatus: open -> closed

assignee: zach.ware
versions: - Python 2.7, Python 3.4
messages: + msg222498
superseder: Update Windows build batch scripts
resolution: fixed
stage: resolved
2014-06-27 16:51:43zach.waresetmessages: + msg221692
2014-06-27 15:50:41BreamoreBoysetnosy: + zach.ware
2014-06-27 15:06:29steve.dowersetmessages: + msg221691
2014-06-27 14:39:43BreamoreBoysetversions: + Python 3.4, Python 3.5, - Python 3.1, Python 3.2
nosy: + BreamoreBoy, steve.dower

messages: + msg221689

components: + Windows
2011-01-30 21:29:04db3lsetnosy: db3l, pitrou, ocean-city
messages: + msg127555
2011-01-30 21:23:44pitrousetnosy: db3l, pitrou, ocean-city
messages: + msg127554
2011-01-30 21:19:43pitrousetnosy: db3l, pitrou, ocean-city
messages: + msg127553
2011-01-30 21:15:08pitrousetnosy: + pitrou
messages: + msg127552
2010-12-07 16:17:13db3lsetnosy: + db3l
messages: + msg123562
2010-12-07 08:49:29ocean-citysetfiles: - py3k_buildbot_error_in_clean_faze.patch
2010-12-07 08:49:01ocean-citysetfiles: + py3k_buildbot_error_in_clean_faze_v2.patch
2010-09-28 16:34:19ocean-citycreate