This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Windows frozen .exe multiprocessing.Queue access is denied exception
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: alex_python_org, davin, jnoller, sbt, vstinner
Priority: normal Keywords:

Created on 2016-01-25 02:54 by alex_python_org, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg258895 - (view) Author: Alex Robinson (alex_python_org) Date: 2016-01-25 02:54
A pyinstaller 3.0 frozen .exe Python 2.7.10 program under Windows 7 that uses a multiprocessing.Queue to send things to a multiprocessing.Process leads to the process getting access-is-denied exceptions on every q.get() call.

And, when the program can't exit. Or leaves a dangling process for every Process.

An unsatisfying fix for this is to put the following code somewhere in the program:

"""
    Do what must be done to make multiprocessing work in .exe files.

    This involves monkey patching multiprocessing.forking under Windows
    so when the a program using a multiprocessing.Process exits,
    there won't be processes left running.

    Hint from   http://stackoverflow.com/questions/33764448/pathos-multiprocessing-pipe-and-queue-on-windows

    .           The bottom line is we get "access is denied" when trying
    .           to get from the multiprocessing.Queue when we're an .exe file.

    .           So, in multiprocessing.forking.duplicate(),
    .           we change 'inheritable' to default to True
    .           from the normal code's False.

"""

import  sys
import  multiprocessing
import  multiprocessing.forking


#
#
#
#
def duplicate(handle, target_process = None, inheritable = True) :
    return(multiprocessing.forking.kludge_to_fix_dangling_processes(handle, target_process, inheritable))

if  (not hasattr(multiprocessing.forking, 'kludge_to_fix_dangling_processes')) and (sys.platform == 'win32') :
    multiprocessing.forking.kludge_to_fix_dangling_processes    = multiprocessing.forking.duplicate
    multiprocessing.forking.duplicate                           = duplicate
msg260896 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-02-26 16:53
Using tools like pyinstaller (and other competing frozen-exe-creating tools) unfortunately complicates things for multiprocessing to understand the environment it's now in.  It is unclear from this description whether this should be regarded as an issue to be addressed in pyinstaller or in multiprocessing itself.  Any further information along these lines would be much appreciated.
msg260992 - (view) Author: Alex Robinson (alex_python_org) Date: 2016-02-29 05:58
Sorry I can't help more than provide a test environment for any fix. I just plucked the "fix" from StackOverflow and it fixed the Q problem on my machine.

It appears, at the least, the multiprocessing code should probably not rely on the default value for the 'inheritable' argument. That argument does sound like one that might be different in the usual case for Win32 and Unix, just because.
msg353067 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-24 10:38
No activity since 2016 and Davin considers that we lack information to debug this issue, so I close this issue as out of date.
History
Date User Action Args
2022-04-11 14:58:26adminsetgithub: 70383
2019-09-24 10:38:51vstinnersetstatus: open -> closed

nosy: + vstinner
messages: + msg353067

resolution: out of date
stage: resolved
2016-02-29 05:58:46alex_python_orgsetmessages: + msg260992
2016-02-26 16:53:50davinsetnosy: + davin
messages: + msg260896
2016-01-30 00:50:15terry.reedysetnosy: + jnoller, sbt
2016-01-25 02:54:44alex_python_orgcreate