Author vstinner
Recipients gregory.p.smith, izbyshev, nanjekyejoannah, pablogsal, pitrou, serhiy.storchaka, vstinner
Date 2018-12-20.16:05:57
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1545321961.08.0.788709270274.issue35537@psf.upfronthosting.co.za>
In-reply-to
Content
> In the end, I think that migrating subprocess to vfork-exec would have more impact for users than adding "fast paths" and have consistent performance regardless of subprocess.Popen arguments (with few exceptions). Please consider it. Thanks!

I'm open to experiment to use vfork() in _posixsubprocess, but I still want to use posix_spawn():

* it's standard
* it's safer
* it can be more efficient

On macOS, posix_spawn() is implemented as a syscall.

Using vfork() can cause new issues: that's why there is a POSIX_SPAWN_USE_VFORK flag (the caller had to explicitly enable it). See also bpo-34663 the history of vfork in posix_spawn() in the glibc. I'm not against using it, but I'm not sure that vfork() can be used in all cases, when all subprocess options are used. Maybe you will end up with the same state: vfork() used or not depending on subprocess.Popen options...


> For example, close_fds is True by default in subprocess, and I don't think there is a reliable way to use posix_spawn() in this case.

Since PEP 446 has been implemented in Python 3.4, I would be interested to revisit close_fds default value: don't close by default... But I'm not sure if it's *safe* to change the default... At least, we may promote close_fds=False in the doc explaining that it's faster and should be safer in the common case.


> Another example is "cwd" parameter: there is no standard posix_spawn file action to change the working directory in the child, so such a seemingly trivial operation would trigger the "slow" fork-exec path.

I don't think that it's a bug that subprocess have different performances depending on the flags. It would be a bug if the behavior would be diffrent. I don't think that it's the case.
History
Date User Action Args
2018-12-20 16:06:01vstinnersetrecipients: + vstinner, gregory.p.smith, pitrou, serhiy.storchaka, izbyshev, pablogsal, nanjekyejoannah
2018-12-20 16:06:01vstinnersetmessageid: <1545321961.08.0.788709270274.issue35537@psf.upfronthosting.co.za>
2018-12-20 16:06:00vstinnerlinkissue35537 messages
2018-12-20 16:05:57vstinnercreate