Created on 2009-07-11 06:08 by sgm, last changed 2015-03-05 18:03 by davin. This issue is now closed.
|msg90405 - (view)||Author: Stuart Mentzer (sgm)||Date: 2009-07-11 06:08|
Freezing apps with multiprocessing on Windows seems to be broken. First, in get_command_line in multiprocessing/forking.py I find that this code: if getattr(sys, 'frozen', False): return [sys.executable, '--multiprocessing-fork'] else: prog = 'from multiprocessing.forking import main; main()' return [_python_exe, '-c', prog, '--multiprocessing-fork'] should be: elif getattr(sys, 'frozen', False) and not WINEXE: return [sys.executable, '--multiprocessing-fork'] else: prog = 'from multiprocessing.forking import main; main()' return [_python_exe, '-c', prog, '--multiprocessing-fork'] in order for the _python_exe set with multiprocessing.set_executable to be used rather than your app's exe. Second, I can then get a working "frozen" package if I include pythonw.exe (and use set_executable to point to it) and a subset of Python's Lib directory that my process needs to call. If this is as intended then it needs to be documented. This may just be a flaw in py2exe. Third, the multiprocessing documentation page description for set_executable has example code with the older setExecutable call.
|msg90646 - (view)||Author: Jesse Noller (jnoller) *||Date: 2009-07-17 22:49|
Thanks for noticing this. Unfortunately, I don't know enough about py2exe/windows to dig into that part. I can fix the low hanging fruit though
|msg90690 - (view)||Author: Stuart Mentzer (sgm)||Date: 2009-07-18 21:13|
Thanks Jesse. If you make broader changes that my patch I am happy to test on Windows and with py2exe if that is helpful. I will try to get the py2exe folks to look at the broader multiprocessing issues I describe as well. If multiprocessing supports freezing on Windows then it would be nice if its docs had a short primer on what you'll actually need to do to get that working with freeze tools. Stuart
|msg90691 - (view)||Author: Jesse Noller (jnoller) *||Date: 2009-07-18 21:16|
Sounds good - I've personally never used freeze on windows, so having some docs and a patch to help make it better for those that do is a definite plus
|msg91416 - (view)||Author: Stuart Mentzer (sgm)||Date: 2009-08-08 05:15|
Further experimentation revealed that freeze_support() works properly and the proposed patch is not necessary or desirable for the general freeze case. In the case of an embedded app that calls set_executable() it seems that the "else" block calling _python_exe should be taken but won't be, so I still think there may be a problem here. In our situation it turned out that an imported module was grabbing the command line arguments before freeze_support() was called and when that was addressed the need for a separate python executable and calling set_executable() was eliminated. I suggest that the documentation for freeze_support() make it clear that it must get called before an application processes the arguments or that the application must allow a --multiprocessing-fork option with a string value so that the initialization continues until freeze_support() is called.
|msg91418 - (view)||Author: Jesse Noller (jnoller) *||Date: 2009-08-08 13:46|
Thanks for following up on this Stuart - I'll update the docs accordingly
|msg98154 - (view)||Author: spongebob (spongebob)||Date: 2010-01-22 17:38|
I can't seem to freeze an app on Windows using multiprocessing.Manager() Are apps that use Managers freezable?? Example: from multiprocessing import freeze_support,Manager if __name__ == '__main__': freeze_support() m=Manager() will throw errors, when run: Process SyncManager-1: Traceback (most recent call last): File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.process" , line 232, in _bootstrap File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.process" , line 88, in run File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.managers ", line 520, in _run_server IOError: [Errno 6] The handle is invalid Traceback (most recent call last): File "<string>", line 5, in <module> File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing", line 9 8, in Manager File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.managers ", line 499, in start EOFError
|msg189783 - (view)||Author: Mark Lawrence (BreamoreBoy) *||Date: 2013-05-21 20:40|
Could someone answer the question posed in msg98154 please.
|msg191520 - (view)||Author: Richard Oudkerk (sbt) *||Date: 2013-06-20 14:40|
I just tried freezing the program from multiprocessing import freeze_support,Manager if __name__ == '__main__': freeze_support() m=Manager() l = m.list([1,2,3]) l.append(4) print(l) print(repr(l)) using cx_Freeze with Python 2.7, and it worked fine: PS> cxfreeze.bat foo.py copying C:\Python27\lib\site-packages\cx_Freeze\bases\Console.exe -> C:\Tmp\dir\dist\foo.exe copying C:\Windows\system32\python27.dll -> C:\Tmp\dir\dist\python27.dll writing zip file C:\Tmp\dir\dist\foo.exe ... PS> dist\foo [1, 2, 3, 4] <ListProxy object, typeid 'list' at 0x206f410>
|msg237281 - (view)||Author: Davin Potts (davin) *||Date: 2015-03-05 18:03|
The original issue now appears addressed in the docs (thanks goes to Stuart and Jesse) though it was not explicitly tracked here as a patch file. The follow-on secondary issue from spongebob (if that is your real name) could not be reproduced by Richard. This issue should have been marked closed->resolved some time ago.
|2015-03-05 18:03:43||davin||set||status: open -> closed|
nosy: + davin
messages: + msg237281
|2013-06-20 14:40:59||sbt||set||messages: + msg191520|
messages: + msg189783
messages: + msg98154
|2009-08-08 13:46:26||jnoller||set||messages: + msg91418|
|2009-08-08 05:15:25||sgm||set||messages: + msg91416|
|2009-07-18 21:16:14||jnoller||set||messages: + msg90691|
|2009-07-18 21:13:45||sgm||set||messages: + msg90690|
|2009-07-17 22:49:35||jnoller||set||messages: + msg90646|
|2009-07-17 22:31:54||ncoghlan||set||assignee: jnoller|
nosy: + jnoller