This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess fails when used as init, vfork() returns EINVAL if PID=1
Type: behavior Stage: commit review
Components: Extension Modules Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: corona10, gregory.p.smith, miss-islington, socketpair, vstinner
Priority: normal Keywords: patch

Created on 2022-03-29 10:54 by socketpair, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 32186 merged gregory.p.smith, 2022-03-30 01:23
PR 32219 merged miss-islington, 2022-03-31 20:42
Messages (7)
msg416252 - (view) Author: Марк Коренберг (socketpair) * Date: 2022-03-29 10:54
The bug introduced here: #35823

If os.gepid() == 1, vfork() does not work and returns EINVAL.

Please add check for the current pid in condition where CPython chooses between fork() and vfork().
msg416316 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-30 01:33
Any possibility that you can test the attached PR as pid 1?
msg416334 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-30 10:47
> Any possibility that you can test the attached PR as pid 1?

Python should be built as a static binary and can be used as the init process on the kernel command line, no? I'm not sure that "static binary" is a requirement, since I'm commonly using init=/usr/bin/bash to fix a broken Linux, and on my Fedora, this program is "dynamically linked".

Who uses Python as pid 1? I'm now curious :-)
msg416416 - (view) Author: Марк Коренберг (socketpair) * Date: 2022-03-31 07:23
Well.
1. We use Python as PID 1. In PID namespace.
2. Further investigation gave information that vfork()+pid=1 actually WORKS. The problem is connected with another weird thing in kernel (undocumented unshare() flags).
3. The logic to try fork() after vfork() has failed is NICE. Please merge. I would also write a message to stderr in case vfork() failed.

P.S. Program may or may not be static in order to work as PID=1.
msg416452 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-31 17:59
Thanks. I had wondered if this was really a pid=1 restriction or not, but I could definitely imagine kernel scenarios where vfork is simply forbidden regardless. There are a variety of policy mechanisms in kernels, mainline Linux or not, that _could_ do that kind of thing.

As much as I'd like to expose that the fallback happened, emitting to stderr isn't friendly and using warnings from this code is complicated so I'm inclined to keep the silent fallback on failure simple as is until someone can demonstrate of it causing a practical problem.

Outside of unusual configurations, if this were ever happening when it is not expected, people observing low subprocess performance could strace and witness the vfork syscall failure.

I'll merge once our CI is happy.
msg416455 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-31 20:42
New changeset 4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835 by Gregory P. Smith in branch 'main':
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
https://github.com/python/cpython/commit/4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835
msg416460 - (view) Author: miss-islington (miss-islington) Date: 2022-03-31 21:09
New changeset 9ed179b07df6ce7432f972f5d069a7c8dee56e79 by Miss Islington (bot) in branch '3.10':
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
https://github.com/python/cpython/commit/9ed179b07df6ce7432f972f5d069a7c8dee56e79
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91307
2022-03-31 23:12:07gregory.p.smithsetstatus: open -> closed
assignee: gregory.p.smith
resolution: fixed
stage: patch review -> commit review
2022-03-31 21:09:58miss-islingtonsetmessages: + msg416460
2022-03-31 20:42:41miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request30295
2022-03-31 20:42:37gregory.p.smithsetmessages: + msg416455
2022-03-31 17:59:44gregory.p.smithsetmessages: + msg416452
2022-03-31 07:23:50socketpairsetmessages: + msg416416
2022-03-30 10:47:43vstinnersetnosy: + vstinner
messages: + msg416334
2022-03-30 01:33:14gregory.p.smithsetmessages: + msg416316
2022-03-30 01:23:35gregory.p.smithsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request30263
2022-03-29 16:06:22gregory.p.smithsettitle: vfork() returns EINVAL if PID=1 -> subprocess fails when used as init, vfork() returns EINVAL if PID=1
type: behavior
components: + Extension Modules, - Library (Lib)
stage: needs patch
2022-03-29 14:31:47corona10setnosy: + gregory.p.smith
2022-03-29 14:28:50corona10setnosy: + corona10
2022-03-29 10:54:10socketpaircreate