Title: faulthandler dumps 'Windows fatal exception: code 0xe06d7363'
Type: behavior Stage: patch review
Components: Windows Versions: Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Fynn Be, haypo, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2017-10-05 09:59 by Fynn Be, last changed 2017-10-09 16:52 by haypo.

Pull Requests
URL Status Linked Edit
PR 3928 closed haypo, 2017-10-09 10:25
PR 3929 merged haypo, 2017-10-09 11:23
Messages (16)
msg303751 - (view) Author: Fynn Be (Fynn Be) Date: 2017-10-05 09:59
c++ extension compiled with MSVC 14
using python 3.6.2 on Windows 10 x64

Whenever a C++ exception is thrown in an extension the faulthandler dumps a traceback to it, even if it is caught.
msg303752 - (view) Author: Fynn Be (Fynn Be) Date: 2017-10-05 10:00
Here is a git repository with a test module and a test script:
msg303755 - (view) Author: Fynn Be (Fynn Be) Date: 2017-10-05 10:15
In the github repository there is 'build.bat' that creates the mscv 14 solution, builds it and runs the ''.
Here is the output:
Windows fatal exception: code 0xe06d7363

Current thread 0x0000462c (most recent call first):
  File "", line 5 in <module>
caught test error: c++ error message

The 'Windows fatal exception: code 0xe06d7363' is only caused by a c++ exception thrown (regardless if it is handled or not). More on the error code is here:

The last line 'caught test error: c++ error message' is generated within the test module, where the error is caught.

In larger projects these tracebacks spam the faulthandler's output, as this is valid code and occurs often (for example for an IndexError raised in a c++ extension to end a for loop).
msg303758 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-05 11:48
The code of the current faulthandler exception handler lives at:



3. Within the Exceptions dialog box, select error 0xE06D7363.
4. Change the value of Action from Stop if not handled to Stop always.

Are you asking to *ignore* all 0xE06D7363 exceptions?

I don't know well Windows exceptions. The latest change was bpo-30557, ignore non-fatal exceptions:

    /* bpo-30557: only log fatal exceptions */
    if (!(code & 0x80000000)) ...
msg303771 - (view) Author: Fynn Be (Fynn Be) Date: 2017-10-05 14:25
> Are you asking to *ignore* all 0xE06D7363 exceptions?
Yes. This error is only indicating that an error was thrown, regardless if it was handled or not. Therefore it should not be treated as fatal, but it is by faulthandler: 'bool(0xE06D7363 & 0x80000000) == True'

>  with the initial "E" standing for "exception" and the final 3 bytes (0x6D7363) representing the ASCII values of "msc"
In this context it seems like a coincidence to me.
msg303773 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-10-05 15:05
We may just need to special case some well known exception codes for MSVC and the CLR, though if they're handled we may be dumping on the first chance and not the second chance handling.

If I get a chance to dig into the docs and code I'll take a look, but it's been years since I really did anything with SEH (apart from the recent filtering change, which was pretty trivial).
msg303888 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-10-07 16:22
Haven't had a chance to test this, but I suspect our change should depend on what happens when a C++ exception is actually unhandled.

In this case, the following sequence has occurred:
* exception is raised
* debug event is raised (if a debugger is attached)
* stack-based handlers get first-chance to ignore the exception
 * there is a stack-based handler that says "let it be raised" and the search stops
* vector-based handlers are called
* stack unwind begins, and when the handler is reached it allows execution to continue

Unfortunately, I don't think there's going to be any difference between the unhandled and handled cases at the point where our handler exists - at worst, the OS is going to say that it is "handled" and its handler will kill the process. The first chance lookup is actually there to continue execution and ignore the exception (possibly after fixing an argument or allocating a new page, etc.).

So I think our choices are:
* report no C++/CLR exceptions via faulthandler
* report all C++/CLR exceptions via faulthandler (current behaviour)

I'm inclined towards changing to the first option. The runtime is going to dump a more useful message than us if the exception makes it that far, and a C++ exception can never be caused by CPython itself.
msg303889 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-10-07 16:27
Okay, just tested taking out the C++ handler and it goes straight to an error report. Not ideal, especially since the error is logged against "python.exe" and "ucrtbase.dll" (which is going to make my life harder when reviewing the submitted crash reports), but since it can only be caused by user code, it's probably better than spamming on every handled exception.

E06D7363 (msc) and E0434352 (CCR) at least should be ignored. I think there's a third, but don't remember what it is right now.
msg303927 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 07:37
On the Internet, I also found the code 0xE0434F4D: 0xe0000000 + 'COM'. Should it be ignored as well?
msg303928 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 07:44
> So I think our choices are:
> * report no C++/CLR exceptions via faulthandler
> * report all C++/CLR exceptions via faulthandler (current behaviour)
> I'm inclined towards changing to the first option. (...)

On the Internet, I found users asking why their favorite application crashed with "Unhandled Exception E0434F4D (e0434fedh) at address 7C81EB33h" or "APPCRASH Exception Code e0434f4d".

While I understand that it's annoying to get a traceback each time an application handles an exception, the faulthandler would leave the user clueless if faulthandler ignore such exception and the exception "kills" the process.

faulthandler registers its exception handler as the *first* handler to be called:

   AddVectoredExceptionHandler(1, faulthandler_exc_handler);

Maybe it shouldn't be the first but the *last* to be called? So an earlier handler could handle the exception, and faulthandler wouldn't log its traceback.

The problem is that I'm confused with "non-error" exceptions (code < 0x80000000), "expected" MSC or COM exceptions, and fatal exceptions like EXCEPTION_INT_DIVIDE_BY_ZERO or EXCEPTION_ACCESS_VIOLATION.

Is it possible to be asked to be called as the last handler, and still be able to log EXCEPTION_ACCESS_VIOLATION? If not, maybe we need two handlers depending on the exception code. One handler called last for non-fatal exceptions, one handler called first for fatal exceptions.

By "Fatal" exceptions, I mean:

    case EXCEPTION_ACCESS_VIOLATION: PUTS(fd, "access violation"); break;
    case EXCEPTION_FLT_DIVIDE_BY_ZERO: PUTS(fd, "float divide by zero"); break;
    case EXCEPTION_FLT_OVERFLOW: PUTS(fd, "float overflow"); break;
    case EXCEPTION_INT_DIVIDE_BY_ZERO: PUTS(fd, "int divide by zero"); break;
    case EXCEPTION_INT_OVERFLOW: PUTS(fd, "integer overflow"); break;
    case EXCEPTION_IN_PAGE_ERROR: PUTS(fd, "page error"); break;
    case EXCEPTION_STACK_OVERFLOW: PUTS(fd, "stack overflow"); break;
msg303939 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 10:31
I wrote the PR 3928 to call AddVectoredExceptionHandler(0, ...) rather than AddVectoredExceptionHandler(1, ...), but it doesn't work as expected.

I tested "faulthandler.enable(); faulthandler._sigsegv()": a traceback is logged. But this code tested the *signal handler* rather than the exception handler. I disabled manually the code of faulthandler.enable() to not install signal handlers anymore: no more traceback is logged.
msg303940 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 10:52
I misunderstood how Windows works. UNIX signals handlers and Windows exception handlers are unrelated. Exception handlers are not called to handle a SIGSEGV signal (raised manually by the process itself, not raised by the Windows kernel).
msg303941 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 11:09
I failed to build with a Python built myself. But I succeeded to recompile a Python extension (_overlapped) in C++ (I removed two functions which caused compilation error) and then added faulthandler-spam/test_module/module.cpp code into _overlapped. So I was able to test C++ code raising a regular extension called by Python.

Sadly, using "AddVectoredExceptionHandler(0, faulthandler_exc_handler);" (instead of "AddVectoredExceptionHandler(1, ...") doesn't solve the issue: the exception is still logged. It seems like the faulthandler exception handler is called before C++ has the opportunity to handle the exception.

So it doesn't seem possible to log *unhandled* C++ exceptions using AddVectoredExceptionHandler() without flooding logs with *handled* C++ extensions.

I now agree with Steve Dower to ignore *all* C++ exceptions in faulthandler.
msg303942 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 11:25
I abandon my intent to log all interesting errors: PR 3929 now always ignore all MSC and COM Windows exceptions.
msg303977 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-10-09 15:30
> Is it possible to be asked to be called as the last handler ...

Unfortunately not, since the stack-based handlers always come after the vector handlers, and C++ handlers written using try/catch will always be stack-based.

It may be interesting for faulthandler to have a stack-based version, so that you can provide the function to call and it will call it inside an exception handler. But I don't think it's that interesting and in any case doesn't need to be in the stdlib.

I wouldn't worry about 0xE0434F4D for now, but if someone comes along with a need for it then we can add it.
msg303985 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-10-09 16:52
New changeset 6e3d6b5dc22cd06d8c4d44a38a8a3415e4bebb16 by Victor Stinner in branch 'master':
bpo-31701: faulthandler: ignore MSC and COM Windows exception (#3929)
Date User Action Args
2017-10-09 16:52:34hayposetmessages: + msg303985
2017-10-09 15:30:56steve.dowersetmessages: + msg303977
2017-10-09 11:25:24hayposetmessages: + msg303942
2017-10-09 11:23:54hayposetpull_requests: + pull_request3904
2017-10-09 11:09:53hayposetmessages: + msg303941
2017-10-09 10:52:21hayposetmessages: + msg303940
2017-10-09 10:31:25hayposetmessages: + msg303939
2017-10-09 10:25:57hayposetkeywords: + patch
stage: patch review
pull_requests: + pull_request3903
2017-10-09 07:44:53hayposetmessages: + msg303928
2017-10-09 07:37:26hayposetmessages: + msg303927
2017-10-07 16:27:40steve.dowersetmessages: + msg303889
2017-10-07 16:22:10steve.dowersetmessages: + msg303888
2017-10-05 15:05:16steve.dowersetmessages: + msg303773
2017-10-05 14:25:07Fynn Besetmessages: + msg303771
2017-10-05 11:48:20hayposetmessages: + msg303758
2017-10-05 10:15:27Fynn Besetmessages: + msg303755
2017-10-05 10:00:29Fynn Besetmessages: + msg303752
2017-10-05 09:59:11Fynn Becreate