This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing not working from egg
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: asksol, davin, eric.araujo, jnoller, simonsteiner
Priority: low Keywords: needs review, patch

Created on 2010-04-26 10:52 by simonsteiner, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
forking.patch simonsteiner, 2010-04-26 12:00
Messages (3)
msg104215 - (view) Author: simon (simonsteiner) Date: 2010-04-26 10:52
testmultiprocessing.py:

def main():
    import multiprocessing

    proc = multiprocessing.Process(target=runhi)
    proc.start()
    proc.join()
    
def runhi():
    print 'hi'
    
if __name__ == "__main__":
    main()

testmultiprocessing.py is inside myegg.egg

set PYTHONPATH=myegg.egg

python -m testmultiprocessing

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python26\lib\multiprocessing\forking.py", line 341, in main
    prepare(preparation_data)
  File "C:\Python26\lib\multiprocessing\forking.py", line 450, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named testmultiprocessing
msg138337 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-14 15:48
I’m not sure this belongs on the Python tracker.
msg235047 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-01-30 19:03
The example demonstrating the issue is reproducible on Windows (tested on Windows 7 64-bit, specifically) with 2.7.9.  Complications arising from how multiprocessing creates new processes on Windows combined with conventions in the import system in 2.7.9 result in the problem being triggered here.

This issue is specific to Windows platform use:  It was not possible to reproduce the issue on OS X nor Linux with 2.7.9.

As played out in the discussion around issue10845 (long since closed), solutions were devised and put into place as part of PEP451, exposed first in Python 3.4.  And so it was not possible to reproduce this issue with default (3.5) on Windows 7.

It is very much appreciated anytime such a nice, clean demonstration code is provided and more than doubly so anytime a proposed patch is also offered.  While the supplied/proposed patch does provide some workaround it is not yet a comprehensive solution and appears to potentially cause complications in other scenarios.  The current best effort at a comprehensive solution is reflected in the changes designed and now implemented in 3.4.

Two aspects to the resolution on this issue:
* For the 2.x branches, it is unlikely that we will see a backport of PEP451 and so a more comprehensive solution might depend upon some other fundamental changes to how multiprocessing operates on Windows platforms but there are no such plans to invest that level of effort into 2.7.  It is still possible that someone might become so motivated about multiprocessing+Windows+2.7 but that seems ever more unlikely.
* For the 3.x branches, we have verified that changes in 3.4 have resulted in a fix for this particular issue.  Overall, marking this as fixed in 3.[4-5].
History
Date User Action Args
2022-04-11 14:57:00adminsetgithub: 52780
2015-01-30 19:03:43davinsetstatus: open -> closed

nosy: + davin
messages: + msg235047

resolution: fixed
stage: needs patch -> resolved
2011-06-14 15:48:57eric.araujosetnosy: + eric.araujo
messages: + msg138337
2010-08-31 11:00:54asksolsetkeywords: + needs review
nosy: + asksol
2010-04-26 12:00:21simonsteinersetfiles: + forking.patch
keywords: + patch
2010-04-26 11:29:35pitrousetnosy: + jnoller
versions: + Python 3.1, Python 2.7, Python 3.2
priority: low
assignee: jnoller
type: behavior
stage: needs patch
2010-04-26 10:52:14simonsteinercreate