classification
Title: test suite: faulthandler signal handler is lost
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: signal module ignores external signal changes
View: 13285
Assigned To: Nosy List: neologix, pitrou, r.david.murray, vstinner
Priority: normal Keywords:

Created on 2013-08-28 16:24 by neologix, last changed 2013-08-29 07:51 by neologix. This issue is now closed.

Messages (7)
msg196387 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-28 16:24
At the beginning of the test suit execution, faulthandler registers its signal handler for fatal signals, as well as SIGUSR1 and SIGALRM.

The problem is that some tests temporary change those handlers, e.g. for EINTR-handling tests.
While they do restore the original handlers after the test, this doesn't work as expected because of an artifact of the signal implementation: when a signal handler isn't registered through the signal module (e.g. directly from C for faulthandler), getsignal() and setsignal() return None/SIG_DFL (depending on whether it was registered before or after the module initialization). Which means that when the test is done, it will restore the default signal handler, and not the faulthandler's one. Thus, any subsequent crash due to this signal won't benefit from faulthandler debug output, see e.g. (issue #18786):
http://buildbot.python.org/all/builders/PPC64%20PowerLinux%203.x/builds/616/steps/test/logs/stdio
"""
[310/379] test_multiprocessing_spawn
make: *** [buildbottest] User defined signal 1
/home/shager/cpython-buildarea/3.x.edelsohn-powerlinux-ppc64/build/Lib/multiprocessing/semaphore_tracker.py:121: UserWarning: semaphore_tracker: There appear to be 4 leaked semaphores to clean up at shutdown
  len(cache))
program finished with exit code 2
elapsedTime=2459.665997
"""

One way to fix this would be to add a WithSignalHandler(signal, handler) context manager to test.support that would replace the signal handler upon entry, and re-register faulthandler's handler upon exit.
msg196390 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-08-28 16:55
See also issue 3948 for another variation of this problem with getsignal/setsignal.
msg196394 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-28 17:50
Or perhaps we could enhance the signal module so that getsignal() return something (i.e. a specific object) which can restore the C signal handler. That would be better than special-casing faulthandler, IMHO.
msg196399 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-28 18:24
> Or perhaps we could enhance the signal module so that getsignal() return something (i.e. a specific object) which can restore the C signal handler. That would be better than special-casing faulthandler, IMHO.

Yes, I thought about that, see http://bugs.python.org/msg146560
The only problem I saw at that time is that we must make sure that the
object is indeed valid, otherwise the signal handler can jump to an
invalid address, and boom.
msg196400 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-28 18:26
> > Or perhaps we could enhance the signal module so that getsignal() return something (i.e. a specific object) which can restore the C signal handler. That would be better than special-casing faulthandler, IMHO.
> 
> Yes, I thought about that, see http://bugs.python.org/msg146560
> The only problem I saw at that time is that we must make sure that the
> object is indeed valid, otherwise the signal handler can jump to an
> invalid address, and boom.

Well, to make sure the object is valid, you can just ensure instances
can't be created from Python (for example, make tp_new raise an error),
and that the internal state can't be mutated.
msg196408 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-08-28 19:46
"One way to fix this would be to add a WithSignalHandler(signal, handler) context manager to test.support that would replace the signal handler upon entry, and re-register faulthandler's handler upon exit."

faulthandler.dump_traceback_later() is registered before each test, in test.regrtest.runtest().

You may reinstall signal handlers before each test, or fix signal.getsignal() and signal.signal(). I don't have a preference.
msg196417 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-28 20:32
I'll tackle #13285 instead.
History
Date User Action Args
2013-08-29 07:51:31neologixsetstatus: open -> closed
2013-08-28 20:32:46neologixsetsuperseder: signal module ignores external signal changes
resolution: duplicate
messages: + msg196417
stage: resolved
2013-08-28 19:46:58vstinnersetmessages: + msg196408
2013-08-28 18:26:52pitrousetmessages: + msg196400
2013-08-28 18:24:40neologixsetmessages: + msg196399
2013-08-28 17:50:22pitrousetnosy: + pitrou
messages: + msg196394
2013-08-28 16:55:20r.david.murraysetnosy: + r.david.murray
messages: + msg196390
2013-08-28 16:24:53neologixcreate