classification
Title: Add Linux's signalfd() to the signal module
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, njs, pitrou
Priority: low Keywords:

Created on 2017-12-29 01:27 by gregory.p.smith, last changed 2018-01-06 07:38 by njs.

Messages (6)
msg309151 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-12-29 01:27
We should add a wrapper for both signalfd() and a function to read and decode the structure from the fd into a dataclass.

The code we need to build such a thing from appears to exist in BSD & MIT licensed form in:

PyPI contains two extension module wrappers of just the system call:
 https://pypi.python.org/pypi/signalfd/
 https://pypi.python.org/pypi/python-signalfd

Why add this Linux specific API to the standard library?  I believe I could use signalfd() within the subprocess module to get rid of the need to busy-loop-poll to see when children have exited on subprocess.Popen.wait() calls with a timeout.  [a proof of concept using the above modules would be a good idea first]
msg309152 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-12-29 01:39
An example subprocess improvement using sigtimedwait() - https://github.com/python/cpython/pull/5035 - led me to believe that signalfd() is superior as it sounds like signalfd() does not require pthread_sigmask (sigprocmask) global state manipulation calls.  I have not confirmed if that is true.
msg309166 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-12-29 09:35
See also https://bugs.python.org/issue12304
msg309167 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-12-29 09:45
Also see how the forkserver module does without it:
https://github.com/python/cpython/blob/master/Lib/multiprocessing/forkserver.py#L146

Reading Jean-Paul's messages in https://bugs.python.org/issue8407, I'm unclear what is required to take advantage of signalfd().  Jean-Paul says """In order to effectively use signalfd(), the signals in question must be blocked, though""".  The signalfd man page says:

"""Normally, the set of signals to  be  received  via the  file descriptor should be blocked using sigprocmask(2), to prevent the signals being handled according to their default dispositions""".

And it's not clear what is meant by that (what happens if you don't block those signals?). Also:

"""As a consequence of the read(2), the signals are consumed, so that they are no longer pending for the process (i.e., will not be caught by signal handlers, and cannot be accepted using sigwaitinfo(2))."""

But how about the converse?  i.e. can a signal be consumed first by a signal handler and the fd not written to at all?

Also see fork() semantics which might (or not) require special handling.

Also see https://bugs.python.org/issue31489 for an issue related to fork(), signals and fds.
msg309196 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-12-29 20:57
My reading of the Linux signalfd man page may be optimistic. :)

Regardless, it'd be nice to have it available in the stdlib so it could be used if deemed useful.  I expect this to only ever be added by someone making use of it in another stdlib module.

As for what multiprocessing.forkserver does, the old manual signal handler and pipe trick is a reasonably well known one.  But a forkserver is not safe to be started when threads exist.  (unlike subprocess)

Signals are process global state, no thread compatible library can rightfully take ownership of a one.
msg309229 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-12-30 12:04
> Regardless, it'd be nice to have it available in the stdlib so it could be used if deemed useful.

Agreed.

> Signals are process global state, no thread compatible library can rightfully take ownership of a one.

But then is the signalfd() idea for subprocess doomed as well?
History
Date User Action Args
2018-01-06 07:38:43njssetnosy: + njs
2017-12-30 12:04:03pitrousetmessages: + msg309229
2017-12-29 20:57:52gregory.p.smithsetpriority: normal -> low

messages: + msg309196
2017-12-29 09:45:29pitrousetmessages: + msg309167
2017-12-29 09:35:07pitrousetnosy: + pitrou
messages: + msg309166
2017-12-29 01:39:44gregory.p.smithsetmessages: + msg309152
2017-12-29 01:27:20gregory.p.smithcreate