classification
Title: Update pickle to take advantage of PEP 451
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: 19700 19946 Superseder:
Assigned To: Nosy List: Arfrever, alexandre.vassalotti, brett.cannon, cameron, eric.snow, larry, ncoghlan, pitrou
Priority: normal Keywords:

Created on 2013-11-22 16:35 by brett.cannon, last changed 2019-03-10 10:30 by ncoghlan.

Messages (5)
msg204645 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-28 07:45
I don't recall the specifics of how we'd talked about making use of module specs in pickle.  I vaguely remember (or misremember <wink>) something related to saving __main__.__spec__.name in the pickle rather than __main__.__name__.  Anyone have anything more concrete than that?  I'm willing to work this out but only with a more specific goal relative to the pickle module.
msg204648 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-28 10:20
The specific proposal was to use __spec__.name when __name__ == "__main__"
to avoid the pickle compatibility issues described in PEP 395 when using
the -m switch.

runpy has to be updated first, though.
msg206427 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-12-17 12:40
Issue 19700 means that runpy now ensures that __main__.__spec__ is set appropriately when __main__ is executed via the import system.

Issue 19946 means that multiprocessing now ensures that __main__ is configured correctly in child processes to reference a properly initialised "fake main" to allow pickle compatibility with classes and functions defined in __main__ outside "if __name__ == '__main__'" guards.

The proposal here is that we make the following changes:

- runpy will ensure that when __main__ is executed via the import system, it will also be aliased in sys.modules as __spec__.name
- if __main__.__spec__ is set, pickle will use __spec__.name rather than __name__ to pickle classes, functions and methods defined in __main__
- multiprocessing is updated appropriately to skip creating __mp_main__ in child processes when __main__.__spec__ is set in the parent process

While I still think this is a reasonable idea, I think it qualifies as a new feature, and hence is better postponed to Python 3.5
msg206450 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-12-17 14:39
So far I agree that this should be postponed to 3.5.
msg337605 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2019-03-10 10:30
Just noting that PEP 499 covers adding modules executed with `-m` to `sys.modules` under their real name in addition to `__main__`.
History
Date User Action Args
2019-03-10 10:30:22ncoghlansetmessages: + msg337605
2015-08-07 03:10:31cameronsetnosy: + cameron
2014-01-07 05:27:47eric.snowunlinkissue18864 dependencies
2013-12-17 14:39:57larrysetmessages: + msg206450
2013-12-17 12:41:16ncoghlansettitle: Update pickle to PEP 451 -> Update pickle to take advantage of PEP 451
2013-12-17 12:40:56ncoghlansetversions: + Python 3.5, - Python 3.4
nosy: + larry

messages: + msg206427

dependencies: + Handle a non-importable __main__ in multiprocessing
2013-11-29 16:01:49brett.cannonsetdependencies: + Update runpy for PEP 451
2013-11-28 10:20:15ncoghlansetmessages: + msg204648
2013-11-28 07:45:11eric.snowsetnosy: + pitrou, alexandre.vassalotti
messages: + msg204645

type: enhancement
stage: test needed -> needs patch
2013-11-23 19:51:28Arfreversetnosy: + Arfrever
2013-11-22 16:37:23brett.cannonlinkissue18864 dependencies
2013-11-22 16:35:18brett.cannoncreate