This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: All os.exec*e variants crash on Windows
Type: crash Stage: resolved
Components: Windows Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: Andreas Bergmeier, eryksun, nicolaje, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2015-02-14 08:45 by nicolaje, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
23462_1.patch steve.dower, 2017-02-05 00:11
Messages (6)
msg235952 - (view) Author: Jeremy Nicola (nicolaje) Date: 2015-02-14 08:45
On Windows 7, using python 3.4.2 32 bits MSVCv.1600 or 3.4.1 64 bits, all the os.exec*e variants crash:

os.execle('C:/Python34/python.exe','Python.exe','-V',{})

os.execve('C:/Python34/python.exe',['python.exe','-V'],{})

os.execlpe('C:/Python34/python.exe','python.exe','-V',{})

os.execvpe('C:/Python34/python.exe',['python.exe','-V'],{})

Without any error message, windows will just open a "Python.exe has stopped working" window, be the scripts run from an interactive shell or invoking python script.py

On the other hand, 
os.execl('C:/Python34/python.exe','Python.exe','-V')

os.execve('C:/Python34/python.exe',['python.exe','-V'])

os.execlpe('C:/Python34/python.exe','python.exe','-V')

os.execvpe('C:/Python34/python.exe',['python.exe','-V'])

will work perfectly.
msg235953 - (view) Author: Jeremy Nicola (nicolaje) Date: 2015-02-14 08:47
You should read:

os.execv('C:/Python34/python.exe',['python.exe','-V'])

os.execlp('C:/Python34/python.exe','python.exe','-V')

os.execvp('C:/Python34/python.exe',['python.exe','-V'])

for the "what works".
msg235975 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-02-14 15:41
Calling nt.execv works with a unicode string because it creates a bytes path via PyUnicode_FSConverter. OTOH, nt.execve uses path_converter, which doesn't convert unicode to bytes on Windows. Thus in posixmodule.c, for the call execve(path->narrow, argvlist, envlist), path->narrow is NULL, and the CRT kills the process due to a bad argument:

    (70.135c): Break instruction exception - code 80000003 (first chance)
    ntdll!LdrpDoDebuggerBreak+0x30:
    00000000`7733cb70 cc              int     3
    0:000> bp ntdll!ZwTerminateProcess
    0:000> bp desktopcrt140!execve
    0:000> g
    ModLoad: 000007fe`fc850000 000007fe`fc868000   C:\Windows\system32\CRYPTSP.dll
    ModLoad: 000007fe`fc550000 000007fe`fc597000   C:\Windows\system32\rsaenh.dll
    ModLoad: 000007fe`fceb0000 000007fe`fcebf000   C:\Windows\system32\CRYPTBASE.dll

    Python 3.5.0a1 (v3.5.0a1:5d4b6a57d5fd, Feb  7 2015, 18:15:14) 
    [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import nt, sys                       
    >>> path = sys.executable
    >>> nt.execve(path, ['python', '-V'], {})

    Breakpoint 1 hit
    DESKTOPCRT140!_execve:
    000007fe`fa5ac43c 4d8bc8          mov     r9,r8
    0:000> r rcx
    rcx=0000000000000000

The first argument is passed in AMD64 register rcx, which you can see is NULL here.

    0:000> g
    Breakpoint 0 hit
    ntdll!NtTerminateProcess:
    00000000`772e1570 4c8bd1          mov     r10,rcx
    0:000> k 8
    Child-SP          RetAddr           Call Site
    00000000`0038f5c8 000007fe`fd11402f ntdll!NtTerminateProcess
    00000000`0038f5d0 000007fe`f54e003a KERNELBASE!TerminateProcess+0x2f
    00000000`0038f600 000007fe`f54e0055 APPCRT140!_invalid_parameter+0x76
    00000000`0038f640 000007fe`fa5abbd8 APPCRT140!_invalid_parameter_noinfo+0x19
    00000000`0038f680 00000000`6aa116a7 DESKTOPCRT140!common_spawnv<char>+0x44
    00000000`0038f6e0 00000000`6aa115a8 python35!os_execve_impl+0xb7
    00000000`0038f720 00000000`6aa9ff3b python35!os_execve+0xa8
    00000000`0038f7d0 00000000`6ab14aed python35!PyCFunction_Call+0xfb

Using a bytes path with nt.execve will work, but it's deprecated on Windows:

    C:\>py -3.5 -Wall
    Python 3.5.0a1 (v3.5.0a1:5d4b6a57d5fd, Feb  7 2015, 18:15:14) 
    [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os, nt, sys
    >>> path = os.fsencode(sys.executable)
    >>> nt.execve(path, ['python', '-V'], {})
    __main__:1: DeprecationWarning: The Windows bytes API has been 
    deprecated, use Unicode filenames instead

    C:\>Python 3.5.0a1

Since bytes paths are deprecated on Windows, these calls should be using wexecv and wexecve.

https://msdn.microsoft.com/en-us/library/431x4c1w%28v=vs.100%29.aspx
msg286301 - (view) Author: Andreas Bergmeier (Andreas Bergmeier) Date: 2017-01-26 09:52
Just hit that bug. Is this really not yet fixed in any newer version?
msg286302 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-01-26 10:00
> Just hit that bug. Is this really not yet fixed in any newer version?

The bug was fixed in Python 3.6.0. It was fixed by Steve Dower in its implementation of the PEP 529, in the change e20c7d8a8187:

-    execv(path_char, argvlist);
+#ifdef HAVE_WEXECV
+    _wexecv(path->wide, argvlist);
+#else
+    execv(path->narrow, argvlist);
+#endif

I'm not sure that it's easy to backport the fix in Python 3.5.

Python 3.6 no walways use Unicode internally to call the Windows API, whereas Python 3.5 used bytes or Unicode depending on the arguments.

So I suggest to upgrade to Python 3.6, or use bytes in the meanwhile.

Would it be ok to close the bug?
msg287024 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-05 00:11
It's straightforward to detect path->narrow == NULL and raise instead of crashing. We can also trivially check path->wide on Windows and raise with a more specific error.

Given it's already fixed for 3.6.0 and there's only one more binary release for 3.5 (and Windows has *very* limited uptake apart from the binary releases), I don't think it's worth making this work given how many versions have been broken in the past.
History
Date User Action Args
2022-04-11 14:58:12adminsetgithub: 67650
2018-02-19 15:41:03steve.dowersetstatus: open -> closed
resolution: out of date
stage: patch review -> resolved
2017-02-05 00:11:37steve.dowersetfiles: + 23462_1.patch
messages: + msg287024

assignee: steve.dower
keywords: + patch
stage: patch review
2017-01-26 10:00:11vstinnersetnosy: + vstinner
messages: + msg286302
2017-01-26 09:52:43Andreas Bergmeiersetnosy: + Andreas Bergmeier
messages: + msg286301
2015-02-14 15:41:31eryksunsetnosy: + eryksun

messages: + msg235975
versions: + Python 3.5
2015-02-14 08:47:44nicolajesetmessages: + msg235953
2015-02-14 08:45:26nicolajecreate