classification
Title: multiprocessing freeze_support needed outside win32
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bbayles, dancol, davin, pitrou
Priority: normal Keywords: patch

Created on 2017-11-27 10:33 by dancol, last changed 2018-01-15 19:44 by bbayles.

Pull Requests
URL Status Linked Edit
PR 5195 open bbayles, 2018-01-15 19:44
Messages (3)
msg307043 - (view) Author: Daniel Colascione (dancol) Date: 2017-11-27 10:33
multiprocessing's freeze_support makes freshly-launched subprocesses integrate correctly until the multiprocessing ecosystem even when the main executable is some application binary instead of a Python interpreter. The documentation and code assume that this support is needed only on win32, but it's equally applicable to POSIX systems using the spawn strategy in combination with systems like cx_freeze.

In particular:

1) The special case in context.py's version of freeze_support() for win32 should be removed

2) The semaphore tracker should correctly tickle the freeze support

3) The documentation should be updated to be platform-neutral
msg307527 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-12-03 22:51
Thank you for reporting this.  Would you like to submit a PR?
msg309886 - (view) Author: bbayles (bbayles) * Date: 2018-01-13 03:11
I ran into this issue, and found references to it on StackOverflow [1] and GitHub [2] as well. I found that the problem applies to both the 'spawn' and 'forkserver' start methods on Linux.

I made an attempt to implement dancol's fix above. (1) and (3) are straightforward, but if there's an elegant way to do the "tickle" in (2) I wasn't able to figure it out?

My branch [3] has something that seems to work with the 'spawn' method and cx_Freeze. Perhaps someone could look at that and tell me if I'm on a reasonable track? If so, I can attempt to extend the method to cover 'forkserver' as well.

[1] https://stackoverflow.com/q/47325297/353839

[2] https://github.com/anthony-tuininga/cx_Freeze/issues/264

[3] bpo-32146-freeze_support">https://github.com/python/cpython/compare/master...bbayles:bpo-32146-freeze_support
History
Date User Action Args
2018-01-15 19:44:39bbaylessetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request5049
2018-01-13 03:11:23bbaylessetnosy: + bbayles
messages: + msg309886
2017-12-03 22:51:59pitrousetversions: - Python 2.7, Python 3.4, Python 3.5, Python 3.8
nosy: + pitrou, davin

messages: + msg307527

stage: needs patch
2017-11-27 10:33:55dancolcreate