Title: signal module ignores external signal changes
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: flox, jdemeyer, martin.panter, neologix, njs, petri.lehtinen, pitrou, r.david.murray, rpcope1, takluyver, vilya, vstinner
Priority: normal Keywords:

Created on 2011-10-28 11:27 by vilya, last changed 2019-01-16 17:43 by jdemeyer.

Messages (18)
msg146553 - (view) Author: Vilya Harvey (vilya) Date: 2011-10-28 11:27
The signal module is oblivious to any changes to the set of installed signal handlers which occur outside of the module. This can happen when a native module changes a signal handler, or when the python interpreter is embedded in another program which installs its own signal handlers.

In this case, saving and restoring a signal handler through python doesn't work correctly. For example, if the SIGINT handler is set externally after the signal module is initialised, the following code will replace the external signal handler with python's default_int_handler:

  handler = signal.getsignal(signal.SIGINT)
  signal.signal(signal.SIGINT, handler)

So it's impossible to reliably save and restore signal handlers through python when they can also be changed outside the python interpreter.

Also, if there's a signal handler installed before the module is initialised, signal.getsignal() will return None for it - making it impossible to restore the correct handler after disabling it.

The reason is that the signal module only checks for existing handlers when it's initialised. The results get stored in the Handlers array, which is then used by all subsequent calls to signal.getsignal(). There are no further checks to see whether the native signal handlers have changed.
msg146560 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-10-28 14:01
> So it's impossible to reliably save and restore signal handlers through 
> python when they can also be changed outside the python interpreter.

signal.getsignal() or signal.signal() return the current/previous handler as a Python function. How could it return a reference to a native (i.e. C) signal handler?
While we could in theory return it as a magic cookie (i.e. the handler's address as returned by sigaction/signal) that can just be passed back to signal.signal(), it would be a bad idea: if the user passes an invalid address, the process will crash when the signal is received.
msg146564 - (view) Author: Vilya Harvey (vilya) Date: 2011-10-28 14:25
Could it return an opaque wrapper object, rather than just the raw address? Something like:

typedef struct _PyNativeSignalHandler {
  sighandler_t handler_func;
} PyNativeSignalHandler;

where the type object doesn't expose any way to read or manipulate the handler_func. Would that work, do you think?
msg146868 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2011-11-02 19:00
> Could it return an opaque wrapper object, rather than just the raw address? Something like:

Are you suggesting that it would return either a Python function object or a wrapper object?
msg146872 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-11-02 19:30
>> Could it return an opaque wrapper object, rather than just the raw
>> address? Something like:
> Are you suggesting that it would return either a Python function object or a
> wrapper object?

I think it would be an ugly kludge. I don't like the idea of passing
around an opaque "blob" just for that purpose (I mean, it's really a
corner case).
So I'm -1.
msg146935 - (view) Author: Vilya Harvey (vilya) Date: 2011-11-03 14:12
Petri: yes, that what I was suggesting.

Charles-François: I'm certainly open to alternatives. Unless I've overlooked something though, the problem is that no workaround is possible at the moment. BTW this came up in the context of a customer script for the software I work on, so it's not just a theoretical problem - at least, not for me. :-)

I guess another solution would be methods to save and restore the native signal handers, e.g. savesignal(signum) & restoresignal(signum). That wouldn't entirely solve the problem - if someone called setsignal() before calling savesignal(), they'd end up in the same situation - but it would at least permit a workaround.
msg262229 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-03-23 02:26
Also, the documentation currently suggests it returns None in these cases, but it actually returns SIG_DFL in at least one case (noticed in Issue 23735).

FWIW Vilya’s opaque object sounds fairly sensible to me. Yes, it is ugly, but one does not look at Unix signals expecting to find beauty :)
msg262247 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-23 08:05
If anyone wants to work on this issue, I suggest to requalify it as a documentation issue. Just explain the behaviour and don't try to implement complex workaround.
msg285831 - (view) Author: Thomas Kluyver (takluyver) * Date: 2017-01-19 18:49
I'd like to make the case for a fix in the code again. Our use case is, I believe, the same as Vilya's. We want to temporarily set a signal handler from Python and then restore the previous handler. This is fairly straightforward for Python handler functions, and SIG_DFL and SIG_IGN, but it breaks if anything has set a C level signal handler.

The opaque wrapper object is a solution that had occurred to me too. Another option would be a context manager implemented in C (I assume context managers can be written in C) which can set one or more signal handlers on entry, and restore them on exit.
msg285842 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-01-19 20:12
IMO the signal handler context manager would be useful (I have existing code where I wrote one that didn't quite work right :).  I suggest you propose this on python-ideas.
msg286264 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017-01-25 15:56
Let me add that this "low-level opaque object" would be rather easy to implement on POSIX systems (I have no clue about other systems such as Windows). I could implement it, but it would be good to have some pre-approval from Python devs that it's a good idea to do this.
msg286266 - (view) Author: Thomas Kluyver (takluyver) * Date: 2017-01-25 16:34
I pitched both the opaque handle and the context manager to python-ideas, but didn't get any responses, positive or negative.
msg286332 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017-01-26 20:32
Here is a proposal for an API:

* getsignal: return the Python-level signal handler (this is an existing function)

* setsignal: set the Python-level signal handler (but not the OS-level signal handler)

* getossignal: get the OS-level signal handler as opaque object

* setossignal: set the OS-level signal handler with an opaque object

Using these primitives, you could implement anything that you want on a higher level.
msg291529 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017-04-12 08:29
Ping? I'll try to implement this in cysignals (which is more difficult than doing it in CPython because I cannot access all internals of the Python signal module).
msg291548 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017-04-12 12:50
I have a preliminary implementation (in the cysignals package) at
msg325700 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-09-19 04:00
Here's another case where this bug bites us:

At startup, Trio checks if SIGINT is currently being handled by Python's default SIGINT handler, and if so it substitutes its own SIGINT handler (which works just like the default SIGINT handler, except with some added magic [1]).

However, this user has a C library that installs its own handler for SIGINT. When this happens, the Python signal.getsignal() function returns stale, incorrect information (claiming that the Python signal handler is still working, even though it isn't), and this causes Trio to do the wrong thing.

Vilya's "magic cookie" approach above is the one that I was going to suggest before I saw this bug :-).

Jeroen's version seems more complicated than necessary to me, and also it doesn't seem to work for my case: I need to check what the current signal handler is and make some decision based on that result. In Jeroen's API, I can see what the Python-level signal handler is, but there's no way to find out whether that signal handler is actually in use or not. Instead, we should make signal.getsignal() do something like:

  c_handler = PyOS_getsig(signalnum);
  if c_handler == the_python_signal_handler:
      # Python is handling this signal; return the Python-level handler
      return Handlers[signalnum].func
  elif c_handler in [SIG_DFL, SIG_IGN]:
      return c_handler
      return OpaqueCookie(c_handler)

msg333765 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-01-16 13:00
> In Jeroen's API, I can see what the Python-level signal handler is, but there's no way to find out whether that signal handler is actually in use or not.

I added support for that in the latest cysignals release. Now you can do

>>> import signal
>>> from cysignals.pysignals import getossignal, python_os_handler
>>> _ = signal.signal(signal.SIGINT, signal.default_int_handler)
>>> getossignal(signal.SIGINT) == python_os_handler

Note that cysignals is POSIX-only for now (it assumes sigaction), but the code could easily be ported to other systems. Ideally it would become part of CPython's signal module.
msg333780 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-01-16 17:43
For reference, the sources for my implementation:
Date User Action Args
2019-01-16 17:43:44jdemeyersetmessages: + msg333780
2019-01-16 13:00:50jdemeyersetmessages: + msg333765
2018-09-19 04:00:05njssetnosy: + njs
messages: + msg325700
2017-04-12 12:50:48jdemeyersetmessages: + msg291548
2017-04-12 08:29:49jdemeyersetmessages: + msg291529
2017-01-26 20:32:55jdemeyersetmessages: + msg286332
2017-01-25 16:34:00takluyversetmessages: + msg286266
2017-01-25 15:56:43jdemeyersetnosy: + jdemeyer
messages: + msg286264
2017-01-19 20:12:05r.david.murraysetnosy: + r.david.murray
messages: + msg285842
2017-01-19 18:49:49takluyversetnosy: + takluyver
messages: + msg285831
2016-04-04 06:26:02rpcope1setnosy: + rpcope1
2016-03-23 08:05:21vstinnersetmessages: + msg262247
2016-03-23 02:26:31martin.pantersetnosy: + martin.panter
messages: + msg262229
2013-08-28 20:38:14pitrousetnosy: + pitrou
2013-08-28 20:32:46neologixlinkissue18869 superseder
2011-11-03 14:12:14vilyasetmessages: + msg146935
2011-11-02 19:30:48neologixsetmessages: + msg146872
2011-11-02 19:00:22petri.lehtinensetmessages: + msg146868
2011-10-29 17:56:22petri.lehtinensetnosy: + petri.lehtinen
2011-10-28 21:34:16floxsetnosy: + flox

components: + Library (Lib)
versions: + Python 3.3, - Python 2.6, Python 3.1
2011-10-28 14:25:10vilyasetmessages: + msg146564
2011-10-28 14:02:14vstinnersetnosy: + vstinner
2011-10-28 14:01:07neologixsetnosy: + neologix
messages: + msg146560
2011-10-28 11:28:24vilyasettitle: signal module in ignores external signal changes -> signal module ignores external signal changes
2011-10-28 11:27:57vilyacreate