Author gregory.p.smith
Recipients Louis Brandy, Piotr Wysocki, cooperlees, drothera, gregory.p.smith, haypo, phantez, pixelbeat
Date 2017-05-22.17:49:51
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1495475391.68.0.479213788745.issue30395@psf.upfronthosting.co.za>
In-reply-to
Content
You cannot safely use Python's os.fork() in a process that has threads.  Because of POSIX.

The CPython interpreter is by definition not async signal safe so no Python code may safely be executed after the fork() system call if the process contained _any_ threads at fork() system call time.

The only reliable recommendation: *Never use os.fork()* (if your process _might ever_ contain any threads now or in the future started by your or any of your dependencies).

The closest thing to a fix to this is a bunch of band-aids to setup atfork functions to clear the state of known locks in the forked child. -- But even that can't guarantee things because that can't know about all locks.

The C standard library can contain locks.  malloc() for example.  Any C/C++ extension modules can contain locks of their own.  The problem isn't solvable within CPython.  Adding a clear of pystate.c's head_mutex after forking makes sense.  That may even get you further.  But you will encounter other problems of the same nature in the future.

related: https://bugs.python.org/issue6721 and https://bugs.python.org/issue16500
History
Date User Action Args
2017-05-22 17:49:51gregory.p.smithsetrecipients: + gregory.p.smith, haypo, pixelbeat, cooperlees, Louis Brandy, drothera, phantez, Piotr Wysocki
2017-05-22 17:49:51gregory.p.smithsetmessageid: <1495475391.68.0.479213788745.issue30395@psf.upfronthosting.co.za>
2017-05-22 17:49:51gregory.p.smithlinkissue30395 messages
2017-05-22 17:49:51gregory.p.smithcreate