classification
Title: Skip decorator for tests requiring manual intervention on Windows
Type: behavior Stage: resolved
Components: Tests, Windows Versions: Python 3.4, Python 3.3, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ezio.melotti Nosy List: brian.curtin, catalin.iacob, ezio.melotti, haypo, loewis, python-dev, sable, skrah, terry.reedy
Priority: normal Keywords: patch

Created on 2011-03-31 20:53 by brian.curtin, last changed 2013-03-14 01:21 by ezio.melotti. This issue is now closed.

Files
File name Uploaded Description Edit
unattended_decorator.diff brian.curtin, 2011-03-31 20:53 review
faulthandler_example.diff brian.curtin, 2011-03-31 20:54 review
nopopup.diff ezio.melotti, 2013-02-20 06:26 review
nopopup2.diff ezio.melotti, 2013-03-04 22:34 review
nopopup3.2.diff ezio.melotti, 2013-03-06 02:21 Patch for 3.2.
Messages (21)
msg132699 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-03-31 20:53
Attached is a patch which adds skip_unless_unattended, which ideally would be used with at least test_faulthandler when running on Windows.

Running the tests on a Windows desktop results in the user having to click through Windows Error Reporting dialogs in order to continue. Build slaves disable or handle WER dialogs in order to continue running without manual intervention. The patch looks in the common registry keys [0,1] to see the status of WER and skips tests when it finds that WER is enabled and would require user intervention.

We may want to additionally hook this up to a regrtest command line parameter to override the registry setting.

Maybe there's a better name for the decorator - I'm not stuck with what I suggested. I consider build slaves as running in "unattended" mode as opposed to a human kicking off the tests.


[0] http://www.spyany.com/program/registry-disable-error-reporting.htm
[1] http://msdn.microsoft.com/en-us/library/bb513638(v=vs.85).aspx
msg132700 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-03-31 20:54
Attached is an example of how this might be used with Lib/test/test_faulthandler.py
msg132704 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-03-31 21:28
Ah yes, I have also issues with the Windows fault handler (the popup). I disabled this popup manually in my Windows XP box.

Instead of skipping the test, I prefer to disable temporary the popup by setting the right registry key (and then restore the previous value or delete the key).

faulthandler causes fatal errors, but test_capi does also test a crash (I don't know which test exactly). I always have to click on the popup to continue to execute the test suite :-(
msg132706 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-03-31 21:33
Ubuntu has also a fault handler: Apport.
https://wiki.ubuntu.com/Apport

Fedora has abrt.
https://fedorahosted.org/abrt/wiki
http://fedoraproject.org/wiki/Features/ABRT

If we should to disable the Windows fault handler, we may also disable these tools.

test_faulthandler already disable another fault handler: it disables the creation of core files. prepare_subprocess() calls setrlimit(RLIMIT_CORE, (0, 0)).
msg133140 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-04-06 14:54
Disabling and re-enabling is another possibility, and it's probably more likely to help in the long run. Most (all?) Windows machines have error reporting enabled unless you mess with the registry manually, so the only place tests would be run with my first patch would be on build slaves.

I don't currently have a Ubuntu or Fedora machine, but I could look into it. I'll write up a patch that disables/re-enables for Windows and see how it works.
msg139705 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-03 20:39
Issue #12481 has been marked as duplicate: "I'm getting the dreaded "python_d.exe has stopped working" popups in test_faulthandler on Windows 7 + VisualStudioPro + "Debug|x64"."
msg139706 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-03 20:41
"Most (all?) Windows machines have error reporting enabled unless you mess with the registry manually"

It is disabled on all Windows buildbots. test_capi does also test a crash (I don't know/remember which test exactly).
msg139707 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-07-03 20:42
For test_capi the patch in #9116 works. For test_faulthandler it
doesn't, unfortunately.
msg139708 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-07-03 21:05
"Instead of skipping the test, I prefer to disable temporary the popup by setting the right registry key (and then restore the previous value or delete the key)."

Do you mean that the user should change/restore the key or that the
Python tests should do that? I think we shouldn't mess with system
settings, even if we restore them.


BTW, it would be a great help to print a warning (also in test_capi)
that the popups are expected.
msg139710 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-07-03 22:00
> Do you mean that the user should change/restore the key
> or that the Python tests should do that?

Python can change temporary the registry value for the duration of the test.
msg148454 - (view) Author: Catalin Iacob (catalin.iacob) * Date: 2011-11-27 18:06
To avoid messing with system registry settings it sounds like WerRegisterRuntimeExceptionModule could also work, at least on Windows7 http://msdn.microsoft.com/en-us/library/windows/desktop1/dd408167%28v=VS.85%29.aspx

There could be a dll which would do nothing when receiving the crash report for expected crashes (test_capi and test_faulthandler). WerRegisterRuntimeExceptionModule doesn't exist in the headers of the SDK that comes with VS2008 but it could be retrieved at runtime with GetProcAddress. It's more work than the registry setting but seems cleaner since it avoids changing system settings.
msg148455 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2011-11-27 18:10
That would certainly be preferable when available on Windows 7. I'll look into how we can incorporate that.

Thanks for the idea!
msg182474 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-02-20 06:24
Here is a proof of concept patch that defines a context manager that disables the crash popups using SetErrorMode [0].  I tested it only on Windows 7 (and Linux, where it's a no-op).  As an example, the patch fixes the crash popup caused by test_capi.

[0]: http://msdn.microsoft.com/en-us/library/windows/desktop/ms680621%28v=vs.85%29.aspx

FWIW I attempted to use WerRegisterRuntimeExceptionModule too, and I was able to access it from ctypes.windll.kernel32.WerRegisterRuntimeExceptionModule on Win7, but then it wanted me to define a DLL with 3 functions, so I tried to find a simpler solution.
msg183504 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-04 22:34
Attached an updated patch:
- renamed the context manager from no_crash_popups to suppress_crash_popup, as suggested by Brian (I also made it singular, because there shouldn't be more than one crash/popup per call);
- used the context manager on all the crashing tests (test_capi and test_faulthandler);
- added documentation;

I tested it on Windows 7 and Linux (where it's a no-op), and according to MSDN it should work even on Windows XP.  If there aren't other comments I'll commit this soon.  If test_capi is also crashing on 2.7/3.2, I will probably backport it there too.
msg183540 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-05 18:33
New changeset 834a451f1cdb by Ezio Melotti in branch '3.3':
#11732: add a new suppress_crash_popup() context manager to test.support.
http://hg.python.org/cpython/rev/834a451f1cdb

New changeset b87123015fb0 by Ezio Melotti in branch 'default':
#11732: merge with 3.3.
http://hg.python.org/cpython/rev/b87123015fb0
msg183547 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-05 19:04
I applied this only on 3.3/3.x because I didn't manage to test on 2.7/3.2.
If test_ctypes and/or other tests are failing there I can backport it, but the patch needs to be tweaked there because it doesn't apply cleanly (and there's no test_faulthandler there).
msg183572 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-06 02:21
Attached an untested patch against 3.2.
On 2.7 and 3.2 there's no test_faulthandler, and the crashing test in test_ctypes seems to be in 3.x only.

Unless I'm missing something:
 1) no crash popups should appear while running the 2.7 test suite;
 2) only test_ctypes should show a crash popup on 3.2 and the attached patch should fix it;

Brian, if you can confirm this I'll go ahead, backport the patch on 3.2, and null merge it on 3.3/default.
msg183681 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-07 16:39
New changeset 6ccefddc13fd by Ezio Melotti in branch '3.3':
#11732: make suppress_crash_popup() work on Windows XP and Windows Server 2003.
http://hg.python.org/cpython/rev/6ccefddc13fd

New changeset 831035bda9b7 by Ezio Melotti in branch 'default':
#11732: merge with 3.3.
http://hg.python.org/cpython/rev/831035bda9b7
msg183683 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-07 16:46
New changeset c0c440dcb8dd by Ezio Melotti in branch '3.2':
#11732: add a new suppress_crash_popup() context manager to test.support that disables crash popups on Windows and use it in test_ctypes.
http://hg.python.org/cpython/rev/c0c440dcb8dd

New changeset 89d62bc81e47 by Ezio Melotti in branch '3.3':
#11732: null merge with 3.2.
http://hg.python.org/cpython/rev/89d62bc81e47

New changeset 7e818490d297 by Ezio Melotti in branch 'default':
#11732: null merge with 3.3.
http://hg.python.org/cpython/rev/7e818490d297
msg183685 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-07 16:51
GetErrorMode wasn't available on XP/2k3, so I changed the patch to use only SetErrorMode.  I also backported it to 3.2, since ctypes has the same crashing test of 3.3/default.
If I got everything right there shouldn't be anymore crash popups while running the tests, however I haven't tested this on 2.7/3.2, so there might be other popups that I missed.  If there are, feel free to reopen the issue.
msg184123 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-14 01:21
I just noticed that regrtest also has a --nowindows flag that uses SetErrorMode (see Lib/test/regrtest.py:490).
The implementation is a bit different:
 1) it uses msvcrt instead of going through ctypes.windll.kernel32;
 2) it specifies SEM_FAILCRITICALERRORS, SEM_NOALIGNMENTFAULTEXCEPT, and SEM_NOOPENFILEERRORBOX in addition to SEM_NOGPFAULTERRORBOX;
 3) it doesn's seem to save and restore the previous error mode;
 4) it has additional calls to CrtSetReportMode and CrtSetReportFile;
 5) it doesn't check the platform and thus raises an ImportError if msvcrt is missing;

Do you think this flag should be removed?
Should I improve my context manager with any of these things?
History
Date User Action Args
2013-03-14 01:21:25ezio.melottisetmessages: + msg184123
2013-03-07 16:51:43ezio.melottisetnosy: + terry.reedy

messages: + msg183685
versions: + Python 3.2
2013-03-07 16:46:24python-devsetmessages: + msg183683
2013-03-07 16:39:10python-devsetmessages: + msg183681
2013-03-06 02:21:07ezio.melottisetfiles: + nopopup3.2.diff

messages: + msg183572
2013-03-05 19:04:41ezio.melottisetstatus: open -> closed
messages: + msg183547

assignee: brian.curtin -> ezio.melotti
resolution: fixed
stage: patch review -> resolved
2013-03-05 18:33:54python-devsetnosy: + python-dev
messages: + msg183540
2013-03-04 22:34:01ezio.melottisetfiles: + nopopup2.diff

messages: + msg183504
2013-02-20 06:26:51ezio.melottisetfiles: + nopopup.diff
2013-02-20 06:26:13ezio.melottisetfiles: - nopopup.diff
2013-02-20 06:25:45ezio.melottisetfiles: + nopopup.diff
2013-02-20 06:24:40ezio.melottisetnosy: + ezio.melotti

messages: + msg182474
versions: + Python 3.4
2011-11-30 14:20:51sablesetnosy: + sable
2011-11-27 18:10:14brian.curtinsetmessages: + msg148455
2011-11-27 18:06:27catalin.iacobsetmessages: + msg148454
2011-11-27 17:42:18catalin.iacobsetnosy: + catalin.iacob
2011-07-03 22:00:17hayposetmessages: + msg139710
2011-07-03 21:05:34skrahsetmessages: + msg139708
2011-07-03 20:42:58skrahsetmessages: + msg139707
2011-07-03 20:41:18hayposetmessages: + msg139706
2011-07-03 20:39:00hayposetnosy: + skrah
messages: + msg139705
2011-04-06 14:54:16brian.curtinsetmessages: + msg133140
2011-03-31 21:33:04hayposetmessages: + msg132706
2011-03-31 21:28:25hayposetmessages: + msg132704
2011-03-31 20:54:35brian.curtinsetfiles: + faulthandler_example.diff
nosy: + loewis, haypo
messages: + msg132700

2011-03-31 20:53:28brian.curtincreate