Author vstinner
Recipients eryksun, jeremy.kloth, jkloth, koobs, miss-islington, nanjekyejoannah, vstinner
Date 2019-10-09.15:06:10
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1570633570.66.0.264924839656.issue37531@roundup.psfhosted.org>
In-reply-to
Content
The initial issue has been fixed in branches 3.7, 3.8 and master. I close the issue. Thanks to everybody who help to make these changes possible and helped to debug regressions.

It's not perfect, regrtest has to workaround bpo-38207 bug, but at least regrtest should no longer runs for several hours or days.



> regrtest should kill a worker process if it runs longer than --timeout seconds.

The main regrtest process now kills a worker process running longer than timeout seconds. In practice, it's timeout x 1.5, to give time to the faulthandler watchdog to kill the process.

If regrtest hits bpo-38207 bug, regrtest takes 2 min 30 sec to exit in the worst case.


> But it means that the main process was stuck for longer than 60 seconds!? The main process is supposed to write an update "running: ..." every 30 seconds. This buildbot is very slow. Maybe 30 and 60 seconds limits are too low?

I changed this timeout from 1 minute to 5 minutes.


> Calling popen.kill() + popen.stdout.close() + popen.stderr.close() from a thread B does not always interrupt popen.communicate() in thread A.

I propose to continue the discussion in bpo-38207 "subprocess: On Windows, Popen.kill() + Popen.communicate() is blocking until all processes using the pipe close the pipe".

I implemented workarounds in regrtest so regrtest doesn't block *forever*: it only waits 30 seconds until a worker thread completes, and I added a timeout of 2 minutes on Python exit of the main regrtest process to prevent blocking in threading._shutdown(), because of the blocked threads.


> I don't understand why you get 2 typeperf.exe process: only the main process should spawn one. Worker processes don't need it and should not spawn it.

I didn't check on my Windows VM if I have the issue. If someone sees more than one typeperf.exe process, please open a separated issue.

Note: typeperf.exe is the correct spelling (typepref.exe is a typo ;-)).
History
Date User Action Args
2019-10-09 15:06:10vstinnersetrecipients: + vstinner, jkloth, jeremy.kloth, koobs, eryksun, miss-islington, nanjekyejoannah
2019-10-09 15:06:10vstinnersetmessageid: <1570633570.66.0.264924839656.issue37531@roundup.psfhosted.org>
2019-10-09 15:06:10vstinnerlinkissue37531 messages
2019-10-09 15:06:10vstinnercreate