This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Albert.Zeyer
Recipients Albert.Zeyer
Date 2017-10-18.16:24:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1508343864.17.0.213398074469.issue31814@psf.upfronthosting.co.za>
In-reply-to
Content
subprocess_fork_exec currently calls fork().

I propose to use vfork() or posix_spawn() or syscall(SYS_clone, SIGCHLD, 0) instead if possible and if there is no preexec_fn. The difference would be that fork() will call any atfork handlers (registered via pthread_atfork()), while the suggested calls would not.

There are cases where atfork handlers are registered which are not save to be called e.g. in multi-threaded environments. In the case of subprocess_fork_exec without preexec_fn, there is no need to call those atfork handlers, so avoiding this could avoid potential problems. It's maybe acceptable if a pure fork() without exec() doesn't work in this case anymore, but there is no reason that a fork()+exec() should not work in any such cases. This is fixed by my proposed solution.

An example case is OpenBlas and OpenMP, which registers an atfork handler which is safe to be called if there are other threads running.
See here:
https://github.com/tensorflow/tensorflow/issues/13802
https://github.com/xianyi/OpenBLAS/issues/240
https://trac.sagemath.org/ticket/22021

About fork+exec without the atfork handlers, see here for alternatives (like vfork):
https://stackoverflow.com/questions/46810597/forkexec-without-atfork-handlers/
History
Date User Action Args
2017-10-18 16:24:24Albert.Zeyersetrecipients: + Albert.Zeyer
2017-10-18 16:24:24Albert.Zeyersetmessageid: <1508343864.17.0.213398074469.issue31814@psf.upfronthosting.co.za>
2017-10-18 16:24:24Albert.Zeyerlinkissue31814 messages
2017-10-18 16:24:23Albert.Zeyercreate