classification
Title: test_strptime hangs sometimes on AMD64 Windows7 SP1 3.x buildbot
Type: Stage:
Components: Windows Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Windows hangs in call to CRT setlocale()
View: 26624
Assigned To: Nosy List: jkloth, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords:

Created on 2015-10-01 12:55 by vstinner, last changed 2016-03-25 23:28 by vstinner. This issue is now closed.

Messages (7)
msg252025 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-01 12:55
The AMD64 Windows7 SP1 3.x buildbot randomly fails. Tests take a lot of time. I don't know if it's a slow hardware/VM, or if the system is very busy. By the way, regrtest is run with -j4 (run 4 tests in parallel).

Example of errors:

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/6773/steps/test/logs/stdio
---
...
[396/399] test_long -- running: test_lib2to3 (102 sec), test_strptime (54 sec)
[397/399] test_source_encoding -- running: test_lib2to3 (102 sec), test_strptime (54 sec)
running: test_lib2to3 (162 sec), test_strptime (114 sec)
[398/399] test_lib2to3 (205 sec) -- running: test_strptime (159 sec)

command timed out: 3900 seconds without output, attempting to kill
program finished with exit code 1
elapsedTime=4929.577000
---


http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/6783/steps/test/logs/stdio
---
...
[392/399] test_wait3 -- running: test_tools (247 sec), test_strptime (770 sec)
[393/399] test_httplib -- running: test_tools (253 sec), test_strptime (776 sec)
[394/399] test_strtod -- running: test_tools (254 sec), test_strptime (777 sec)
[395/399] test_fork1 -- running: test_tools (255 sec), test_strptime (777 sec)
[396/399] test_xmlrpc -- running: test_tools (263 sec), test_strptime (785 sec)
[397/399] test_multiprocessing_main_handling -- running: test_tools (271 sec), test_strptime (794 sec)
running: test_tools (331 sec), test_strptime (854 sec)
[398/399] test_tools (337 sec) -- running: test_strptime (861 sec)

command timed out: 3900 seconds without output, attempting to kill
program finished with exit code 1
elapsedTime=5068.979000
---
msg252031 - (view) Author: Jeremy Kloth (jkloth) * Date: 2015-10-01 14:58
The system is a dedicated Quad CPU @2.66GHz with 8GB RAM and running the tests on an SSD.  I doubt it is the hardware.

Please note, it is the only 64-bit Windows buildbot AND the only multi-core Windows buildbot.  So it catches lots of things that most others do not.

Last chance I had to find the missing test, it was test__locale.

Also note that 3.5 is passing just fine, so it is something particular to the 3.x branch.
msg252032 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-01 15:46
By the way, I'm surprised how slow is this buildbot. On Linux, running the whole test suite (399 tests) takes less than 5 minutes with -j4 on my PC (Intel i7-2600 @ 3.40GHz with 12 GB of RAM).

Is Python much slower on Windows? Or the buildbot CPU is *much* slower than mine?
msg252033 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2015-10-01 15:58
Jeremy Kloth wrote:
> Please note, it is the only 64-bit Windows buildbot

That's not true anymore; in fact, all but David Bolen's XP and Windows 7 bots are 64-bit.  Yours is the one one in the 'stable' set, but that classification is a bit out of date for the whole set.  Current 64-bit Windows bots are yours, David Bolen's Windows8 and Windows10 bots, and my Windows8.1 Non-Debug bot.

> AND the only multi-core Windows buildbot.

This is still true, though.

Victor Stinner wrote:
> Is Python much slower on Windows?

I think the test suite is significantly slower on Windows; I'm not certain about Python itself.  It routinely takes well over 20 minutes on my Windows machines.  The fact that the bot in question compiles in Debug configuration may also be playing into it; Debug config links with the debug CRT, which enables a bunch of extra checks at the CRT level.
msg252036 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-10-01 16:13
On Linux, I also run the Python test suite with Python compiled in
debug mode. I'm forcing the -O0 flag to disable *all* compilation
optimization. All assertions are kept, etc.

Maybe the slowness comes from CRT checks, maybe.
msg252038 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2015-10-01 16:36
I was curious, so I checked:

C:\Users\Zachary\code\hg.python.org\3.5>python.bat -m test.pystone
Running Debug|x64 interpreter...
Pystone(1.2) time for 50000 passes = 1.14142
This machine benchmarks at 43805 pystones/second

C:\Users\Zachary\code\hg.python.org\3.5>python.bat -m test.pystone
Running Release|x64 interpreter...
Pystone(1.2) time for 50000 passes = 0.421115
This machine benchmarks at 118732 pystones/second

C:\Users\Zachary\code\hg.python.org\3.5>python.bat -m test.pystone
Running PGUpdate|x64 interpreter...
Pystone(1.2) time for 50000 passes = 0.281218
This machine benchmarks at 177798 pystones/second

These were run on a Windows 8.1 Virtualbox VM on a Mac.  For reference, here's the results for the host machine using python3.5.0 from python.org:

$ python3 -m test.pystone
Pystone(1.2) time for 50000 passes = 0.413982
This machine benchmarks at 120778 pystones/second
msg262461 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-25 23:28
It looks like the issue was a bug in setlocale() and that it was fixed in Windows! See the issue #26624.
History
Date User Action Args
2016-03-25 23:28:43vstinnersetstatus: open -> closed
superseder: Windows hangs in call to CRT setlocale()
resolution: duplicate
messages: + msg262461
2015-10-01 16:36:09zach.waresetmessages: + msg252038
2015-10-01 16:13:32vstinnersetmessages: + msg252036
2015-10-01 15:58:23zach.waresetmessages: + msg252033
2015-10-01 15:46:45vstinnersetmessages: + msg252032
2015-10-01 14:58:55jklothsetnosy: + jkloth
messages: + msg252031
2015-10-01 12:55:09vstinnercreate