Message204570
The creation of slave_popen is not protected by a lock, and so dummy_thread() may spawn a new process (master) at the same time than the main process (slave). If it occurs at the same time, the master may inherit a pipe of the slave process used internally by the subprocess module to send the exception to the parent process if a Python exception occurs in the child process.
The inheritance of pipes has been partially fixed in Python 3.2 with the new function _posixsubprocess.cloexec_pipe() which uses pipe2() function to atomically create a pipe with the O_CLOEXEC flag set.
The problem has been fixed complelty in Python 3.4 with the PEP 446: all files are now created non inheritable by default, and atomic functions to set the "non inheritable" flag are used when available.
You posted a Python 2.7 script, so I suppose that you are stuck at Python 2. In this case, you can workaround the issue by using a lock around the creation of any subprocess. To fix your example, just surround slave_popen creation with "with lock: ..." or lock.acquire()/lock.release(), as you did for master_popen.
I propose to convert this issue to a documentation issue: we should add a warning explaining that spawning processes in more than one thread can lead to such race condition and hang one or more threads. And suggest to use a lock to workaround such issue.
See also the atfork proposition which includes such lock:
http://bugs.python.org/issue16500
--
Your script has other issues:
- you should pass a file open in write mode for stdout/stderr
- you should close open(os.devnull) files, or open them outside the loop
- you should join the thread, or use a daemon thread. If you don't join threads, you will quickly read the limit of the number of threads
- you can use shell=True to avoid spawning two process to run sleep (or simply use time.sleep :-))
- you need a synchronization between the two threads to ensure that master_popen is created before trying to kill it |
|
Date |
User |
Action |
Args |
2013-11-27 10:18:06 | vstinner | set | recipients:
+ vstinner, neologix, sbt, owenlin |
2013-11-27 10:18:06 | vstinner | set | messageid: <1385547486.08.0.125377854179.issue19809@psf.upfronthosting.co.za> |
2013-11-27 10:18:06 | vstinner | link | issue19809 messages |
2013-11-27 10:18:04 | vstinner | create | |
|