Message357447
Yeah, the case of pickle + __main__ is an awkward one. [1] However, the approach taken by multiprocessing isn't the right one for subinterpreters.
Multiprocessing operates under 2 design points that do not apply to subinterpreters:
* every process is running in the same __main__ module (sans the "script" part)
* pickle is a critical part of data-passing
For spawn, multiprocessing automatically runs the original __main__ module in each newly spawned process. [2] Note that it runs it there with __name__ set to __mp_main__ (rather than __main__), to keep the "script" part from running.
Subinterpreters could be made to work like this [3] but in reality they are more like processes created using the subprocess module.
I do not expect that to change. However, there is room for add opt-in support for rerunning __main__ in a subinterpreter, or helpers to accomplish the same. We can address such opt-in support or helpers in a separate issue later. For now we are focusing on the fundamentals (at least in the context of PEP 554).
[1] Note that the problem is only with the __main__ module. For other modules pickle does the right thing.
[2] https://github.com/python/cpython/blob/master/Lib/multiprocessing/spawn.py#L287
[3] I expect we will see subinterpreters supported in the multiprocessing module just like threads are supported. |
|
Date |
User |
Action |
Args |
2019-11-25 17:14:04 | eric.snow | set | recipients:
+ eric.snow, crusaderky, LewisGaul |
2019-11-25 17:14:04 | eric.snow | set | messageid: <1574702044.59.0.115890250733.issue37292@roundup.psfhosted.org> |
2019-11-25 17:14:04 | eric.snow | link | issue37292 messages |
2019-11-25 17:14:03 | eric.snow | create | |
|