This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: os.execve puts process to background on windows
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, anacrolix, eric.smith, eryksun, loewis, piotr.dobrogost, r.david.murray, steve.dower, techtonik, tim.golden, trs, zach.ware
Priority: normal Keywords:

Created on 2010-07-03 11:29 by techtonik, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (12)
msg109176 - (view) Author: anatoly techtonik (techtonik) Date: 2010-07-03 11:29
os.execve() is said to replace current process with new program. Unfortunately, when you try to call script that contains os.execve() on windows - that script spawns background process and control is immediately returned to the calling program. Does it behave the same on Unix?

Is there any way to replace current process on Windows so that references to calling parent process are not lost and it could wait for execution to complete?
msg109179 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-03 12:16
on Windows, exec() does not really replace the current process. It creates a new process (with a new pid), and exits the current one.

Hence the calling program only sees that the script has terminated.

I don't see any easy solution on Windows, except than using subprocess.Popen(), and exit the script when the subprocess terminates.
msg109182 - (view) Author: anatoly techtonik (techtonik) Date: 2010-07-03 13:07
Does that mean that windows doesn't allow process replacement at all?

I remember the time then game NoCD loaders were somehow able to load, patch and execute main program in their address space.
msg110463 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-07-16 17:10
I believe it's true that Windows does not offer process replacement. I'm sure you could perform some tricks by essentially writing your own loader, but the practical answer is no. It might be worth looking into how cygwin implements exec().
msg169174 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-08-27 00:11
This looks like the answer to the cygwin question, assuming things haven't changed:

http://cygwin.com/ml/cygwin-developers/2001-02/msg00106.html

Basically, it does the same thing as Python, except that a special return code is reported by the execing process to signal to whoever created the execing process that an exec has occurred.

Would it be at all useful to add such a return code?
msg194379 - (view) Author: Piotr Dobrogost (piotr.dobrogost) Date: 2013-08-04 14:22
This is unexpected and makes people wonder what's going on. See http://stackoverflow.com/q/7004687/95735 and http://stackoverflow.com/q/7264571/95735.
msg221031 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-19 21:51
I've changed the nosy list according to the experts index for the os module and Windows, sorry if I've named anyone I shouldn't have.
msg325682 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-09-18 21:03
Python doesn't go out of its way to fake a cross-platform POSIX environment in Windows, complete with emulated fork and exec. I don't recommend using nt.execv[e] or the os.exec* functions in Windows. Their behavior is almost always undesired. I'd be in favor of deprecating them and eventually removing them. Use subprocess instead.
msg325688 - (view) Author: Thomas Sibley (trs) Date: 2018-09-18 22:17
What about using _execv*() and friends, as documented at <https://docs.microsoft.com/en-gb/cpp/c-runtime-library/reference/execvp-wexecvp?view=vs-2017>?

They appear to use the process overlay feature of _spawnv*() et al., as described at <https://docs.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functions?view=vs-2017> and <https://docs.microsoft.com/en-gb/cpp/c-runtime-library/process-and-environment-control?view=vs-2017>.
msg325689 - (view) Author: Thomas Sibley (trs) Date: 2018-09-18 22:27
It seems like it would be very nice to provide compat if its possible with a minimum of effort.

At the very least, a doc patch to os.exec*() seems warranted.  I expected to find notes on platform compat in the doc and was pleasantly surprised to see the indication of Windows support!  Only later did I find out that it is not meaningfully supported.  :-(
msg325690 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-09-18 22:42
nt.execv[e] wraps calling the C runtime _wexecv[e] function. The other os.exec* functions are in turn based on these calls. As to spawn, for Windows _P_OVERLAY just means to exit the current process. The source code is published, so you can see how simple the implementation is in ucrt\exec\spawnv.cpp:

    if (mode == _P_OVERLAY)
    {
        // Destroy ourselves:
        _exit(0);
    }

If I recall correctly, in the early 80s, Microsoft's C runtime was used for both MS-DOS and Xenix (Unix) development. One assumes that the overlay option was actually useful in Xenix.
msg325692 - (view) Author: Thomas Sibley (trs) Date: 2018-09-18 22:52
Thanks for taking the time to explain.  I was just discovering the same myself experimentally.  The MS docs I linked to seem a little misleading. orz

In that case, do you think a doc patch for os.exec* adding a more nuanced take on Windows support would be considered?  I'd be glad to write it.
History
Date User Action Args
2022-04-11 14:57:03adminsetgithub: 53394
2018-09-18 22:52:38trssetmessages: + msg325692
2018-09-18 22:42:06eryksunsetmessages: + msg325690
2018-09-18 22:27:15trssetmessages: + msg325689
2018-09-18 22:17:30trssettype: behavior ->
messages: + msg325688
2018-09-18 21:03:09eryksunsetstatus: open -> closed

type: behavior

nosy: + eryksun
messages: + msg325682
resolution: third party
stage: resolved
2018-09-18 19:34:56BreamoreBoysetnosy: - BreamoreBoy
2018-09-18 19:32:41trssetnosy: + trs
2014-06-19 21:51:50BreamoreBoysetnosy: + tim.golden, BreamoreBoy, loewis, zach.ware, steve.dower
messages: + msg221031
2013-08-04 14:22:31piotr.dobrogostsetmessages: + msg194379
2013-08-04 13:58:51anacrolixsetnosy: + anacrolix
2013-08-04 13:49:49piotr.dobrogostsetnosy: + piotr.dobrogost
2012-08-27 00:11:15r.david.murraysetnosy: + r.david.murray

messages: + msg169174
versions: + Python 3.4, - Python 2.6, Python 2.7, Python 3.2
2010-07-16 17:10:01eric.smithsetnosy: + eric.smith
messages: + msg110463
2010-07-03 13:07:37techtoniksetmessages: + msg109182
2010-07-03 12:16:34amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg109179
2010-07-03 11:29:33techtonikcreate