classification
Title: subprocess crash under Windows compatibility mode
Type: crash Stage: patch review
Components: Windows Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ma Lin, Segev Finer, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2018-02-11 04:27 by Ma Lin, last changed 2018-03-02 01:46 by Ma Lin.

Files
File name Uploaded Description Edit
crash32818.zip Segev Finer, 2018-03-01 12:06
Pull Requests
URL Status Linked Edit
PR 5948 open Segev Finer, 2018-03-01 12:07
Messages (6)
msg311980 - (view) Author: Ma Lin (Ma Lin) * Date: 2018-02-11 04:27
Reproduce:
Right click python.exe -> properties -> compatibility tab -> enable compatibility mode (e.g. Windows 7)

Then run this test will get a segmentfault: test_multiprocessing_main_handling.py

CPython 3.6 is Ok.
msg311982 - (view) Author: Ma Lin (Ma Lin) * Date: 2018-02-11 04:54
Here is two traceback messages:

D:\git\cpython\PCbuild\win32>python.exe -X dev d:\git\cpython\Lib\test\test_multiprocess
ing_main_handling.py
ssssssssssssssssssssssssssFFFFFFFFFFFFF
======================================================================
FAIL: test_basic_script (__main__.SpawnCmdLineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\git\cpython\Lib\test\test_multiprocessing_main_handling.py", line 162, in test_basic_script
    self._check_script(script_name)
  File "d:\git\cpython\Lib\test\test_multiprocessing_main_handling.py", line 156, in _check_script
    rc, out, err = assert_python_ok(*run_args, __isolated=False)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 151, in assert_python_ok

    return _assert_python(True, *args, **env_vars)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 137, in _assert_python
    res.fail(cmd_line)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 79, in fail
    err))
AssertionError: Process return code is 3221225477
command line: ['D:\\git\\cpython\\PCbuild\\win32\\python.exe', '-X', 'faulthandler', '-E
', 'C:\\Users\\anima\\AppData\\Local\\Temp\\tmplgfds7rc\\script.py', 'spawn']

stdout:
---

---

stderr:
---
Windows fatal exception: access violation

Current thread 0x00001930 (most recent call first):
  File "D:\git\cpython\lib\multiprocessing\popen_spawn_win32.py", line 48 in __init__
  File "D:\git\cpython\lib\multiprocessing\context.py", line 322 in _Popen
  File "D:\git\cpython\lib\multiprocessing\process.py", line 112 in start
  File "D:\git\cpython\lib\multiprocessing\pool.py", line 241 in _repopulate_pool
  File "D:\git\cpython\lib\multiprocessing\pool.py", line 176 in __init__
  File "D:\git\cpython\lib\multiprocessing\context.py", line 119 in Pool
  File "C:\Users\anima\AppData\Local\Temp\tmplgfds7rc\script.py", line 27 in <module>
---

======================================================================
FAIL: test_basic_script_no_suffix (__main__.SpawnCmdLineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\git\cpython\Lib\test\test_multiprocessing_main_handling.py", line 168, in test_basic_script_no_suffix
    self._check_script(script_name)
  File "d:\git\cpython\Lib\test\test_multiprocessing_main_handling.py", line 156, in _check_script
    rc, out, err = assert_python_ok(*run_args, __isolated=False)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 151, in assert_python_ok

    return _assert_python(True, *args, **env_vars)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 137, in _assert_python
    res.fail(cmd_line)
  File "D:\git\cpython\lib\test\support\script_helper.py", line 79, in fail
    err))
AssertionError: Process return code is 3221225477
command line: ['D:\\git\\cpython\\PCbuild\\win32\\python.exe', '-X', 'faulthandler', '-E
', 'C:\\Users\\anima\\AppData\\Local\\Temp\\tmpkovn5nfz\\script', 'spawn']

stdout:
---

---

stderr:
---
Windows fatal exception: access violation

Current thread 0x00001d00 (most recent call first):
  File "D:\git\cpython\lib\multiprocessing\popen_spawn_win32.py", line 48 in __init__
  File "D:\git\cpython\lib\multiprocessing\context.py", line 322 in _Popen
  File "D:\git\cpython\lib\multiprocessing\process.py", line 112 in start
  File "D:\git\cpython\lib\multiprocessing\pool.py", line 241 in _repopulate_pool
  File "D:\git\cpython\lib\multiprocessing\pool.py", line 176 in __init__
  File "D:\git\cpython\lib\multiprocessing\context.py", line 119 in Pool
  File "C:\Users\anima\AppData\Local\Temp\tmpkovn5nfz\script", line 27 in <module>
---
msg312056 - (view) Author: Ma Lin (Ma Lin) * Date: 2018-02-12 14:35
segmentfault conditions:
64bit Windows + 32bit Python + Compatibility Mode enabled.

some infomation:
1, 64bit Python doesn't segmentfault.
2, 3.7.0a3 is ok, 3.7.0a4 segmentfaults.
3, unrelated to Windows SDK, switch from 1709 SDK to 1703 SDK also segmentfaults.
msg312108 - (view) Author: Ma Lin (Ma Lin) * Date: 2018-02-13 08:22
Git bisect says this is the first bad commit:
https://github.com/python/cpython/commit/b2a6083eb0384f38839d3f1ed32262a3852026fa
msg313055 - (view) Author: Ma Lin (Ma Lin) * Date: 2018-02-28 10:55
3.7.0 beta2 still segmentfault.
Finer, please have a look.

segmentfault conditions:
64bit Windows 10 (1709) + 32bit Python 3.7.0 Beta2 + Compatibility Mode enabled.
(64bit Python doesn't segmentfault)
msg313094 - (view) Author: Segev Finer (Segev Finer) * Date: 2018-03-01 12:06
Windows seems to dereference lpStartupInfo->lpAttributeList, despite it being NULL since we had no attributes to set. Smells like a Windows bug in the compatibility shim, since it doesn't crash in non compatibility mode.

Not setting EXTENDED_STARTUPINFO_PRESENT when lpStartupInfo->lpAttributeList is NULL will fix this. Not sure about lpStartupInfo->cb, I think Windows only checks it's large enough.

Also attached is a crash dump with symbols and used source revision of a reproduction of this.

Stack Trace:
00 00fee690 5ad47c1c AcLayers!NS_ElevateCreateProcess::APIHook_CreateProcessW+0x110
01 00fee740 5ad45f79 python38_d!_winapi_CreateProcess_impl+0x15c [cpython\modules\_winapi.c @ 1046]
02 00fee798 5ad78e75 python38_d!_winapi_CreateProcess+0x79 [cpython\modules\clinic\_winapi.c.h @ 326]
03 00fee7f8 5ad77b64 python38_d!_PyMethodDef_RawFastCallKeywords+0x245 [cpython\objects\call.c @ 651]
04 00fee81c 5aed8f9f python38_d!_PyCFunction_FastCallKeywords+0x84 [cpython\objects\call.c @ 730]
05 00fee854 5aee038f python38_d!call_function+0x15f [cpython\python\ceval.c @ 4532]
06 00fee8f4 5ad7b336 python38_d!_PyEval_EvalFrameDefault+0x59bf [cpython\python\ceval.c @ 3128]
07 00fee92c 5ad77ce4 python38_d!function_code_fastcall+0xe6 [cpython\objects\call.c @ 283]
08 00fee994 5ad79be7 python38_d!_PyFunction_FastCallDict+0x134 [cpython\objects\call.c @ 322]
09 00fee9d8 5ad79aa6 python38_d!_PyObject_FastCallDict+0xc7 [cpython\objects\call.c @ 98]
0a 00feea14 5ae0b8f7 python38_d!_PyObject_Call_Prepend+0xf6 [cpython\objects\call.c @ 904]
0b 00feea50 5ae10a97 python38_d!slot_tp_init+0x47 [cpython\objects\typeobject.c @ 6597]
0c 00feea74 5ad7a28e python38_d!type_call+0x187 [cpython\objects\typeobject.c @ 949]
0d 00feeae0 5aed91d1 python38_d!_PyObject_FastCallKeywords+0x37e [cpython\objects\call.c @ 199]
0e 00feeb18 5aedc95b python38_d!call_function+0x391 [cpython\python\ceval.c @ 4576]
History
Date User Action Args
2018-03-02 01:46:24Ma Linsettitle: multiprocessing segmentfault under Windows compatibility mode -> subprocess crash under Windows compatibility mode
2018-03-01 12:07:24Segev Finersetkeywords: + patch
stage: patch review
pull_requests: + pull_request5716
2018-03-01 12:06:40Segev Finersetfiles: + crash32818.zip

messages: + msg313094
2018-02-28 10:55:59Ma Linsetnosy: + Segev Finer
messages: + msg313055
2018-02-13 08:22:54Ma Linsetmessages: + msg312108
2018-02-12 14:35:06Ma Linsetmessages: + msg312056
2018-02-11 04:54:04Ma Linsetmessages: + msg311982
2018-02-11 04:27:54Ma Lincreate