Polling should be avoided when it's possible. For subprocess.wait(), we can do something with signals (SIGCHLD and/or SIGCLD).

sigtimedwait() can be used to wait a the "a child process terminated" with a timeout, but not wait a specific process (so we may use a loop). sigtimedwait() doesn't call the signal handler, and so it changes the behaviour if the parent process has a signal handler for SIGCHLD/SIGCLD.

If sigtimedwait() is not available, we may use a signal handler. For example, we can use the "wakeup fd" tool of the signal module. Problem: the parent program may already have such handler / use "wakeup fd". We should at least restore the previous signal handler when we are done.

On Linux, the clone() syscall allows the change the signal send by the child to the parent when it terminates. It's also possible to choose to not send a signal when at child exit... But I don't think that subprocess uses such options ;-)

Why not use signalfd() when available?

> For subprocess.wait(), we can do something with signals (SIGCHLD and/or SIGCLD).

There's just one problem: SIGCHLD is ignored by default, which means that sigwait and friends won't return when a child exits.
Well, it actually works on recent Linux kernels, but POSIX makes no such guarantee, and it's at least known to fail on Solaris (see Dave Butenhof's comment):
http://www.multithreadedprogramming.info/sigwait-ing-for-sigchld

To be portable, we would need to set a handler for SIGCHLD, which has the following problems:
- you have to do that from the main thread
- it impacts every thread
- it will make syscalls fail with EINTR
- once you've changed SIGCHLD setting, you can't go back to the original semantic (setting it to SIG_IGN again will prevent children from becoming zombies, and waitpid wait until all children exited and will fail with ECHILD)

Note that even if it does work, there's a problem in multi-threaded programs, because the signal must be blocked by all the threads...

But since we use it with a timeout, we could also consider that this will work on systems that allow ignore signals to be catched by sigtimedwait, and it will wait the full timeout on other systems. I don't know if that's acceptable.

> Why not use signalfd() when available?

It suffers from the same issue, and it's Linux-specific (sigwait and friends are POSIX).

Note that exposing sigtimedwait is probably useful anyway, and I'd like to work on a patch.
Note that I'm not sure that exposing sigtimedwait is necessary (I don't think that the info field is going to be used by Python applications): how about just adding an optional timeout argument to signal_sigwait?

> To be portable, we would need to ...

Antoine is right: we don't have to be portable. We can write an "optimized" implementations (without polling) for a specific OS, even for a specific version of an OS (e.g. like Linux kernel >= 2.6.22 for signalfd).

I like the idea of signalfd(), but I don't know exactly how it works. Should we block the signal? What happens when we unblock the signal? It would be nice if the signal handler is called on unblock, because it would not change the current behaviour. Is it possible to block a signal in all threads? pthread_sigmask() blocks signals in the current thread, the manual page of sigprocmask() has a funny comment: "The use of sigprocmask() is unspecified in a multithreaded process; see pthread_sigmask(3)."

Extract of signalfd() manual page: "Normally,  the  set of signals to be received via the file descriptor should be blocked using sigprocmask(2), to prevent the signals being handled according to their default dispositions."

Is SIGCHLD only raised once at child process exit? "SIGCLD would be delivered constantly (unless blocked) while any child is ready to be waited for." according to http://lwn.net/Articles/414618/

> There's just one problem: SIGCHLD is ignored by default,
> which means that sigwait and friends won't return when a child exits.

sigwait() is not impacted by the associated signal handler, but sigwait() only works if the signal is blocked (e.g. by pthread_sigmask):

"If no signal in set is pending at the time of the call, the thread is suspended until one or more becomes pending. The signals defined by set will been blocked at the time of the call to sigwait(); otherwise the behaviour is undefined."
http://pubs.opengroup.org/onlinepubs/007908799/xsh/sigwait.html

Example (for Python 3.3):
--------------------------
from signal import *
import subprocess

signum = SIGCHLD
process = subprocess.Popen("sleep 1", shell=True)
print("Wait %s..." % signum)
pthread_sigmask(SIG_BLOCK, [signum])
sigwait([signum])
pthread_sigmask(SIG_UNBLOCK, [signum])
print("done")
process.wait()
--------------------------

Same question than signalfd(): how can we block a signal in all threads (including C threads, e.g. _tkinter event looop thread)? Use sigprocmask()?

sigwait() removes the signal from the list of pending signals, so the signal handler will not be called.

> Note that exposing sigtimedwait is probably useful anyway,
> and I'd like to work on a patch.

See also issue #8407 for sigtimedwait() and signalfd() in Python.

---

sigprocmask(), sigwait() and signals in general seem to behave differently on each OS, so anyway, we cannot write a single portable implementation to solve this issue. If we cannot write a reliable non-polling implementation for an OS, you should use the polling implementation instead (which *is* reliable).

> Antoine is right: we don't have to be portable. 

We don't have to, but writing one POSIX-conformant solution is better than writing N OS-specific solutions, no? That's what POSIX is about.

> Should we block the signal?

Yes.

> What happens when we unblock the signal?

If you've read from the FD, nothing, since it consumes the pending signals. If you haven't, since signal_pthread_sigmask checks for pending signals, I guess that the handler will be called upon unblock. But signalfd is designed as an alternative to handlers, so I don't think this makes much sense, and if a SIGCHLD handler is setup, it's likely to perform a waitpid(-1, WNOHANG), which will screw up our waiting anyway...

> Is it possible to block a signal in all threads?

Not portably.

> sigwait() is not impacted by the associated signal handler, but sigwait() only works if the signal is blocked (e.g. by pthread_sigmask):

The point I was making is precisely that blocking the signal is not enough on some kernels: when the signal is ignored, it will sometimes not wakeup threads waiting on sigwait.

> sigprocmask(), sigwait() and signals in general seem to behave differently on each OS

They behave correctly as long as they're used in a POSIX-conformant way. To sum up, those problems are:
- since SIGCHLD is ignored by default, some kernels won't wake up threads waiting on sigwait (it works on Linux, don't know for *BSD kernels)
- there's not portable way to block signals in all threads.
As a consequence, there will be cases where sigtimedwait or select on a signalfd will wait until the end of the timeout.

> See also issue #8407 for sigtimedwait() and signalfd() in Python.

You didn't commit the signalfd part?
Whay do you think of sigtimedwait?
Expose it as-is, or just add an optional timeout option to sigwait?

(I should not answer in this issue, but in #8407)

> > See also issue #8407 for sigtimedwait() and signalfd() in Python.
>
> You didn't commit the signalfd part?

Not yet because I would like to provide something to decode the data written into the signalfd file descriptor (msg135438), the signalfd_siginfo structure.

> Whay do you think of sigtimedwait?

It would like to expose it (msg137071, you should read sigtimedwait, not sigwaitinfo :-)). I started to work on a patch, but it requires a siginfo_t structure, and I didn't finish my patch. I will retry later.

> Expose it as-is, or just add an optional timeout option to sigwait?

I prefer thin wrappers: sigwaitinfo() is more than just a timeout argument, there is also the signal info argument.

"On BSDs and OS X, you can use kqueue with EVFILT_PROC+NOTE_EXIT to do exactly that. No polling required. Unfortunately there's no Linux equivalent."
http://stackoverflow.com/questions/1157700/how-to-wait-for-exit-of-non-children-processes/7477317#7477317

An example:
http://doc.geoffgarside.co.uk/kqueue/proc.html

On Linux, it possible to watch processes using a netlink socket:
http://www.outflux.net/blog/archives/2010/07/01/reporting-all-execs/

Example:
http://users.suse.com/~krahmer/exec-notify.c

Python binding (written in Cython) for proc connector:
http://debathena.mit.edu/trac/browser/trunk/debathena/debathena/metrics/debathena/metrics/connector.pyx

There is just a minor limitation: you must be root (CAP_NET_ADMIN) to use this interface...

Honestly, I think the extra complexity and non-portability isn't worth it.

> Honestly, I think the extra complexity and non-portability isn't worth it.

That's what I think too.
If we want to avoid polling, there's another approach:
- fork() a first time
- fork() in the first child
- exec() in the second child
- in the first child, call waitpid() and then write() the return code to
a fd
- in the parent, wait on the fd using select() or poll()

For the PEP 446 (non inheritable files and sockets), it was discussed to
write a helper similar to what Antoine proposes, but to only inherit a few
handles instead all inherit all (inheritable) handles.

The new asyncio module doesn't have this performance issue: it allows to wait asynchronously for the process exit without busy loop.

Right now, there is no high-level API for that, but it is very likely that Python 3.4 final will provide a simple proc.wait() method for that. See #20400 and related Tulip issue:
http://code.google.com/p/tulip/issues/detail?id=115

On Unix, the default implementation sets an handler for SIGCHLD signal which calls waitpid(pid, WNOHANG) on all processes to detect process exit. But it has also a faster implementation which calls waitpid(-1, WNOHANG) only once.

asyncio uses signal.set_wakeup_fd() to wake up its event loop when it gets a signal.

Charles-François wrote:
> Honestly, I think the extra complexity and non-portability isn't worth it.

I agree. And any change may break the backward compatibility, because signal handling is tricky and many detail are platform specific.

asyncio is well designed and solves this issue in a portable way. On Windows, RegisterWaitWithQueue() is used with an overlapped object and a proactor event loop to wait for the process exit.

I leave the issue open until all the new subprocess code is merged into Tulip and Python asyncio.

> Right now, there is no high-level API for that, but it is very likely
> that Python 3.4 final will provide a simple proc.wait() method for
> that.

We aren't supposed to merge new features in 3.4 anymore. I know Tulip
uses a separate repo, but you should only merge bug fixes IMO.

(if you want an exemption, you should ask Larry)

I am trying to be conservative in changing existing Tulip APIs, but I hope to get an exemption from Larry for the "convenience" process API that we are currently adding in Tulip issue 115 (http://code.google.com/p/tulip/issues/detail?id=115).

I expect to be pretty lenient when it comes to asyncio, as it has no installed base yet and is marked provisional.  Also it has a lot of eyes on it right now, so I'm kind of assuming the vetting process for changes at this late date is getting a lot of scrutiny.

Let me know when you have something specific to go in pls.

> The new asyncio module doesn't have this performance issue
> On Unix, the default implementation sets an handler for SIGCHLD signal which 
> calls waitpid(pid, WNOHANG) on all processes to detect process exit. But it has 
> also a faster implementation which calls waitpid(-1, WNOHANG) only once.

But that is still a busy loop, no?
My understanding was that the goal of this ticket was to figure out a strategy to get rid of that.

> But that is still a busy loop, no?

No, it's not. asyncio uses a selector which waits for events on file
descriptors. It uses signal.set_wakeup_fd() which writes
asynchronously on a file descriptor. So asyncio is suspended until a
file descriptor gets data.

Ok, the new asyncio.subprocess module has been merged.

Use asyncio.create_subprocess_exec/shell to create a subprocess and then use asyncio.wait_for(proc.wait(), timeout) to wait for the exit of the process with a timeout. The wait is asynchronous thanks to asyncio internals.

I close the issue. I consider it fixed with the asyncio option. As I wrote, I don't think that it's possible to fix it in subprocess in a portable way without breaking backward compatibility. asyncio is new and so there is no risk of breaking the backward compatibility.