Author sbt
Recipients Giovanni.Bajo, avian, bobbyi, gregory.p.smith, jcea, lesha, neologix, nirai, pitrou, sbt, sdaoden, vstinner
Date 2012-01-23.21:23:21
SpamBayes Score 1.23233e-09
Marked as misclassified No
Message-id <1327353804.31.0.0419255276351.issue6721@psf.upfronthosting.co.za>
In-reply-to
Content
Attached is a patch (without documentation) which creates an atfork module for Unix.

Apart from the atfork() function modelled on pthread_atfork() there is also a get_fork_lock() function.  This returns a recursive lock which is held whenever a child process is created using os.fork(), subprocess.Popen() or multiprocessing.Process().  It can be used like

    with atfork.get_fork_lock():
        r, w = os.pipe()
        pid = os.fork()
        if pid == 0:
            try:
                os.close(r)
                # do something with w
            finally:
                os._exit(0)
        else:
            os.close(w)

    # do something with r

This prevents processes forked by other threads from accidentally inheriting the writable end (which would potentially cause EOF to be delayed when reading from the pipe).  It can also be used to eliminate the potential race where you create an fd and then set the CLOEXEC flag on it.

The patch modifies Popen() and Process.start() to acquire the lock when they create their pipes.  (A race condition previously made Process.sentinel and Process.join() potentially unreliable in a multithreaded program.)

Note that using the deprecated os.popen?() and os.spawn?() functions can still cause accidental inheritance of fds.

(I have also done a hopefully complete patch to multiprocessing to optionally allow fork+exec on Unix -- see Issue 8713.)
History
Date User Action Args
2012-01-23 21:23:25sbtsetrecipients: + sbt, gregory.p.smith, jcea, pitrou, vstinner, nirai, bobbyi, neologix, Giovanni.Bajo, sdaoden, avian, lesha
2012-01-23 21:23:24sbtsetmessageid: <1327353804.31.0.0419255276351.issue6721@psf.upfronthosting.co.za>
2012-01-23 21:23:23sbtlinkissue6721 messages
2012-01-23 21:23:23sbtcreate