This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author eric.snow
Recipients LewisGaul, crusaderky, eric.snow
Date 2019-11-25.17:14:03
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1574702044.59.0.115890250733.issue37292@roundup.psfhosted.org>
In-reply-to
Content
Yeah, the case of pickle + __main__ is an awkward one. [1]  However, the approach taken by multiprocessing isn't the right one for subinterpreters.

Multiprocessing operates under 2 design points that do not apply to subinterpreters:

* every process is running in the same __main__ module (sans the "script" part)
* pickle is a critical part of data-passing

For spawn, multiprocessing automatically runs the original __main__ module in each newly spawned process. [2]  Note that it runs it there with __name__ set to __mp_main__ (rather than __main__), to keep the "script" part from running.  
Subinterpreters could be made to work like this [3] but in reality they are more like processes created using the subprocess module.

I do not expect that to change.  However, there is room for add opt-in support for rerunning __main__ in a subinterpreter, or helpers to accomplish the same.  We can address such opt-in support or helpers in a separate issue later.  For now we are focusing on the fundamentals (at least in the context of PEP 554).


[1] Note that the problem is only with the __main__ module.  For other modules pickle does the right thing.
[2] https://github.com/python/cpython/blob/master/Lib/multiprocessing/spawn.py#L287
[3] I expect we will see subinterpreters supported in the multiprocessing module just like threads are supported.
History
Date User Action Args
2019-11-25 17:14:04eric.snowsetrecipients: + eric.snow, crusaderky, LewisGaul
2019-11-25 17:14:04eric.snowsetmessageid: <1574702044.59.0.115890250733.issue37292@roundup.psfhosted.org>
2019-11-25 17:14:04eric.snowlinkissue37292 messages
2019-11-25 17:14:03eric.snowcreate