Issue38263
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-09-24 10:36 by vstinner, last changed 2022-04-11 14:59 by admin.
Messages (8) | |||
---|---|---|---|
msg353064 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2019-09-24 10:36 | |
On Windows, the multiprocessing DupHandle.detach() method has race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE). Error on duplicate(): Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 107, in spawn_main new_handle = reduction.duplicate(pipe_handle, File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 79, in duplicate return _winapi.DuplicateHandle( PermissionError: [WinError 5] Access is denied Example: bpo-34714 Error on detach(): Traceback (most recent call last): File "<string>", line 1, in <module> File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 117, in spawn_main exitcode = _main(fd) File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 127, in _main self = reduction.pickle.load(from_parent) File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\connection.py", line 951, in rebuild_pipe_connection handle = dh.detach() File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 133, in detach self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE) PermissionError: [WinError 5] Access is denied Example: bpo-34513 |
|||
msg362460 - (view) | Author: Pedro Algarvio (s0undt3ch) * | Date: 2020-02-22 11:20 | |
Any possible workaround for this issue? I seem to be consistingly hitting this issue. |
|||
msg362466 - (view) | Author: Eryk Sun (eryksun) * ![]() |
Date: 2020-02-22 13:44 | |
> I seem to be consistingly hitting this issue. It will help with test development if you can provide a minimal example that reliably reproduces the problem. In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED (5). Assuming the process handles have the required PROCESS_DUP_HANDLE access, it's most likely the case that the underlying NtDuplicateObject system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC000010A). For example: import ctypes ntdll = ctypes.WinDLL('ntdll') from subprocess import Popen, PIPE from _winapi import GetCurrentProcess, TerminateProcess from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE) TerminateProcess(p._handle, 0) Try to duplicate the process handle into the terminated process: >>> source = GetCurrentProcess() >>> target = handle = p._handle >>> try: ... DuplicateHandle(source, handle, target, ... 0, False, DUPLICATE_SAME_ACCESS) ... except: ... status = ntdll.RtlGetLastNtStatus() ... print(f'NTSTATUS: {status & (2**32-1):#010x}') ... raise ... NTSTATUS: 0xc000010a Traceback (most recent call last): File "<stdin>", line 2, in <module> PermissionError: [WinError 5] Access is denied |
|||
msg362471 - (view) | Author: Pedro Algarvio (s0undt3ch) * | Date: 2020-02-22 18:40 | |
What I'm able to copy from the console(though I doubt I'm getting all of the traceback): Traceback (most recent call last): File "<string>", line 1, in <module> File "c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py", line 106, in spawn_main exitcode = _main(fd) File "c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py", line 116, in _main self = pickle.load(from_parent) File "c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\connection.py", line 942, in rebuild_pipe_connection handle = dh.detach() File "c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\reduction.py", line 128, in detach self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE) PermissionError: [WinError 5] Access is denied Pedro Algarvio @ Phone A sábado, 22/02/2020, 13:44, Eryk Sun <report@bugs.python.org> escreveu: > > Eryk Sun <eryksun@gmail.com> added the comment: > > > I seem to be consistingly hitting this issue. > > It will help with test development if you can provide a minimal example > that reliably reproduces the problem. > > In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED > (5). Assuming the process handles have the required PROCESS_DUP_HANDLE > access, it's most likely the case that the underlying NtDuplicateObject > system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC000010A). For > example: > > import ctypes > ntdll = ctypes.WinDLL('ntdll') > from subprocess import Popen, PIPE > from _winapi import GetCurrentProcess, TerminateProcess > from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS > > p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE) > TerminateProcess(p._handle, 0) > > Try to duplicate the process handle into the terminated process: > > >>> source = GetCurrentProcess() > >>> target = handle = p._handle > >>> try: > ... DuplicateHandle(source, handle, target, > ... 0, False, DUPLICATE_SAME_ACCESS) > ... except: > ... status = ntdll.RtlGetLastNtStatus() > ... print(f'NTSTATUS: {status & (2**32-1):#010x}') > ... raise > ... > NTSTATUS: 0xc000010a > Traceback (most recent call last): > File "<stdin>", line 2, in <module> > PermissionError: [WinError 5] Access is denied > > ---------- > nosy: +eryksun > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue38263> > _______________________________________ > |
|||
msg377296 - (view) | Author: Yan Ren (pemryan) | Date: 2020-09-22 03:24 | |
append Nosy List. |
|||
msg389832 - (view) | Author: Eryk Sun (eryksun) * ![]() |
Date: 2021-03-30 15:36 | |
spawn_main() could handle a PermissionError by checking the exit code. If the parent has terminated already, then simply exit quietly. _winapi.PROCESS_QUERY_LIMITED_INFORMATION would need to be defined. For example: def spawn_main(pipe_handle, parent_pid=None, tracker_fd=None): """Run code specified by data received over a pipe.""" assert is_forking(sys.argv), "Not forking" if sys.platform == 'win32': import msvcrt import _winapi if parent_pid is not None: source_process = _winapi.OpenProcess( _winapi.SYNCHRONIZE | _winapi.PROCESS_DUP_HANDLE | _winapi.PROCESS_QUERY_LIMITED_INFORMATION, False, parent_pid) else: source_process = None try: new_handle = reduction.duplicate( pipe_handle, source_process=source_process) fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY) exitcode = _main(fd, source_process) except PermissionError: if (source_process is None or _winapi.GetExitCodeProcess(source_process) == _winapi.STILL_ACTIVE): raise exitcode = 1 else: from . import resource_tracker resource_tracker._resource_tracker._fd = tracker_fd exitcode = _main(pipe_handle, os.dup(pipe_handle)) sys.exit(exitcode) |
|||
msg390229 - (view) | Author: Jesvi Jonathan (jesvi22j) | Date: 2021-04-05 09:10 | |
File "c:/Users/jesvi/Documents/GitHub/Jesvi-Bot-Telegram/scripts/main.py", line 144, in thread_test p.start() File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start self._popen = self._Popen(self) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen return Popen(process_obj) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__ reduction.dump(process_obj, to_child) File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle '_thread.lock' object Traceback (most recent call last): File "<string>", line 1, in <module> Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main new_handle = reduction.duplicate(pipe_handle, File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate new_handle = reduction.duplicate(pipe_handle, File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate return _winapi.DuplicateHandle( OSError: [WinError 6] The handle is invalid return _winapi.DuplicateHandle( OSError: [WinError 6] The handle is invalid |
|||
msg390683 - (view) | Author: Leonardo Rick (leonardo2) | Date: 2021-04-10 01:07 | |
Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main exitcode = _main(fd, parent_sentinel) File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 126, in _main self = reduction.pickle.load(from_parent) File "C:\Program Files\Python39\lib\multiprocessing\connection.py", line 967, in rebuild_pipe_connection handle = dh.detach() File "C:\Program Files\Python39\lib\multiprocessing\reduction.py", line 131, in detach return _winapi.DuplicateHandle( PermissionError: [WinError 5] Acesso negado |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:20 | admin | set | github: 82444 |
2021-04-10 01:07:14 | leonardo2 | set | nosy:
+ leonardo2 messages: + msg390683 versions: + Python 3.9, - Python 3.8 |
2021-04-05 09:10:36 | jesvi22j | set | versions:
- Python 3.9, Python 3.10 nosy: + jesvi22j messages: + msg390229 type: behavior -> compile error |
2021-03-30 16:04:59 | vstinner | set | nosy:
- vstinner |
2021-03-30 15:36:07 | eryksun | set | type: behavior stage: needs patch messages: + msg389832 versions: + Python 3.10, - Python 3.7 |
2020-09-22 03:24:51 | pemryan | set | nosy:
+ pemryan messages: + msg377296 |
2020-02-22 18:40:50 | s0undt3ch | set | messages: + msg362471 |
2020-02-22 13:44:17 | eryksun | set | nosy:
+ eryksun messages: + msg362466 |
2020-02-22 11:20:43 | s0undt3ch | set | nosy:
+ s0undt3ch messages: + msg362460 |
2019-09-24 10:36:39 | vstinner | create |