classification
Title: Real segmentation fault handler
Type: Stage:
Components: Versions:
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Rhamphoryncus, amaury.forgeotdarc, belopolsky, ggenellina, haypo, pitrou, skip.montanaro
Priority: normal Keywords: patch

Created on 2008-09-30 01:10 by haypo, last changed 2010-05-31 19:16 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
segfault-3.patch haypo, 2008-12-10 02:50 Raise MemoryError on SIGSEGV and ArithmeticError on SIGFPE
fault.py haypo, 2008-12-10 18:34
Messages (12)
msg74060 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-09-30 01:10
I would like to be able to catch SIGSEGV in my Python code! So I 
started to hack Python trunk to support this feature. The idea is to 
use a signal handler which call longjmp(), and add setjmp() at 
Py_EvalFrameEx() enter.

See attached ("small") patch: segfault.patch

Example read.py with the *evil* ctypes module of invalid memory read:
------------------- 8< --------------
from ctypes import string_at

def fault():
    text = string_at(1, 10)
    print("text = {0!r}".format(text))

def test():
    print("test: 1")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: 2")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: end")

def main():
    test()

if __name__ == "__main__":
    main()
------------------- 8< --------------

Result:
------------------- 8< --------------
$ python read.py
test: 1
sizeof()=160
ooops!
segmentation fault
test: 2
sizeof()=160
ooops!
segmentation fault
test: end
------------------- 8< --------------

Example bug1.py of a stack overflow:
----------
loop = None,
for i in xrange(10**5):
    loop = loop, None
----------

Result:
----------
$ python -i bug1.py

(((((((((...Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: segmentation fault

----------

Python is able to restore a valid state (stack/heap) after a 
segmentation fault and raise a classical Python exception (I choosed 
MemoryError, but it could be a specific exception).

On my computer (Ubuntu Gutsy/i386), each segfault_frame takes 
sizeof(sigjmpbuf) + sizeof(void*) = 160 bytes, allocated on the stack. 
I don't know if it's huge or not, but that will limit the number of 
recursive calls. The feature can be optional if we add a configure 
option and some #ifdef/#endif. A dedicated stack is needed to be call 
the signal handler on stack overflow error. I choosed 4 KB, but since 
I only call longjmp(), smaller stack might also works.

Does other VM support such feature? JVM, Mono, .NET, etc. ?

I had the idea of catching SIGSEGV after reading the issue 1069092 
(stack overflow because of too many recursive calls).
msg74066 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-09-30 08:54
Did you consider using PyOS_CheckStack for this?
Currently there is only a Windows implementation, but it seems that the
primitives you use in your patch could form a Unix version.
msg74067 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-09-30 09:02
@amaury.forgeotdarc: It looks like PyOS_CheckStack() is only 
implemented for Windows. It uses alloca() + __try/__except + 
_resetstkoflw(). The GNU libc nor Linux kernel don't check stack 
pointer on alloca(), it's just $esp += <alloca argument>. Using 
alloca() you may also be able to able outside the stack to move your 
stack pointer to the heap or another memory mapping. PyOS_CheckStack() 
doesn't really protect the stack: if a function use alloca() or a 
similar construction like « void test(int size) { char 
allocated_on_the_stack[size]; ... } », you will not catch this error.

PyOS_CheckStack() only checks one type of error: stack overflow. It 
doesn't check invalid memory read / write (see my first example, 
read.py).
msg74068 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-09-30 09:06
Note: my patch can be adapted to catch SIGFPE (divison by zero or 
other math error). For int/long types, Python avoids divison by zero, 
but for code written in C ("external modules"), Python is unable to 
catch such errors. Eg. see last imageop issue: i was possible to 
generate many divisions by zero.
msg74089 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-09-30 18:12
Oops, my patch was broken. I forgot to install the fault handler! Here 
is a new version of the patch which also catch SIGFPE: raise an 
ArithmeticError.
msg77478 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-12-10 02:50
New patch:
 - limit memory footprint: use a static buffer to store the frames, 
with a maximum of MAXDEPTH frames (default: MAXDEPTH=100)
 - if there are more than MAXDEPTH frames, jump to the frame MAXDEPTH 
on error (it's like a truncated traceback)
 - don't call segfault_exit() in PyEval_EvalFrameEx() if 
segfault_enter() was not called

On Ubuntu Gutsy, for MAXDEPTH=100 the memory footprint is 19996 bytes. 
I think that it's small, but it's possible to reduce the footprint by 
using less frames or disable the alternative stack (needed for stack 
overflow errors).
msg77479 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-12-10 02:54
Oh, another change in segfault-3.patch:
 - disable signal handler before the first call to segfault_enter() 
and the last call to segfault_exit()

About the memory footprint: it would be possible to use variable size 
buffer using malloc() and then realloc(). But static buffers are 
easier to use, and I don't want to play with malloc() while I'm 
handling a segmentation fault or stack overflow :-)
msg77563 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2008-12-10 18:34
fault.py: catch two segfaults in Python and C contexts.
msg78745 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-01-02 01:30
As mentioned in python-dev, the patch would be more suitable for
inclusion if it was changed to simply print a stack trace and bail out,
rather than try to resume execution of the Python program.
msg95055 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2009-11-09 01:43
My idea was rejected on python-dev mailing list. But I'm unable to
write a patch to dump a backtrace on segfault. Anyway it would be a
complelty different patch (and so a different issue). So I prefer to
close this (old) issue.
msg95095 - (view) Author: Adam Olsen (Rhamphoryncus) Date: 2009-11-09 18:51
That's fine, but please provide a link to the new issue once you create it.
msg106804 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-05-31 19:16
> That's fine, but please provide a link to the new issue once you create it.

Done: issue #8863.
History
Date User Action Args
2010-05-31 19:16:32hayposetmessages: + msg106804
2009-11-09 18:51:32Rhamphoryncussetmessages: + msg95095
2009-11-09 01:43:24hayposetstatus: open -> closed

messages: + msg95055
2009-01-02 01:30:28pitrousetnosy: + pitrou
messages: + msg78745
2008-12-11 03:38:42skip.montanarosetnosy: + skip.montanaro
2008-12-11 00:13:23belopolskysetnosy: + belopolsky
2008-12-10 22:54:24ggenellinasetnosy: + ggenellina
2008-12-10 18:58:53Rhamphoryncussetnosy: + Rhamphoryncus
2008-12-10 18:34:36hayposetfiles: + fault.py
messages: + msg77563
2008-12-10 02:54:19hayposetmessages: + msg77479
2008-12-10 02:50:14hayposetfiles: - segfault-2.patch
2008-12-10 02:50:08hayposetfiles: + segfault-3.patch
messages: + msg77478
2008-09-30 18:13:06hayposetfiles: - segfault.patch
2008-09-30 18:13:00hayposetfiles: + segfault-2.patch
messages: + msg74089
2008-09-30 09:06:02hayposetmessages: + msg74068
2008-09-30 09:02:53hayposetmessages: + msg74067
2008-09-30 08:54:24amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg74066
2008-09-30 01:10:11haypocreate