Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pickle to take advantage of PEP 451 #63901

Open
brettcannon opened this issue Nov 22, 2013 · 5 comments
Open

Update pickle to take advantage of PEP 451 #63901

brettcannon opened this issue Nov 22, 2013 · 5 comments
Labels
stdlib Python modules in the Lib dir topic-importlib topic-multiprocessing type-feature A feature request or enhancement

Comments

@brettcannon
Copy link
Member

BPO 19702
Nosy @brettcannon, @ncoghlan, @pitrou, @larryhastings, @avassalotti, @cameron-simpson, @ericsnowcurrently
Dependencies
  • bpo-19700: Update runpy for PEP 451
  • bpo-19946: Handle a non-importable main in multiprocessing
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2013-11-22.16:35:18.731>
    labels = ['type-feature', 'library']
    title = 'Update pickle to take advantage of PEP 451'
    updated_at = <Date 2019-03-10.10:30:22.147>
    user = 'https://github.com/brettcannon'

    bugs.python.org fields:

    activity = <Date 2019-03-10.10:30:22.147>
    actor = 'ncoghlan'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2013-11-22.16:35:18.731>
    creator = 'brett.cannon'
    dependencies = ['19700', '19946']
    files = []
    hgrepos = []
    issue_num = 19702
    keywords = []
    message_count = 5.0
    messages = ['204645', '204648', '206427', '206450', '337605']
    nosy_count = 8.0
    nosy_names = ['brett.cannon', 'ncoghlan', 'pitrou', 'larry', 'alexandre.vassalotti', 'cameron', 'Arfrever', 'eric.snow']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'needs patch'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue19702'
    versions = ['Python 3.5']

    @brettcannon brettcannon added the stdlib Python modules in the Lib dir label Nov 22, 2013
    @ericsnowcurrently
    Copy link
    Member

    I don't recall the specifics of how we'd talked about making use of module specs in pickle. I vaguely remember (or misremember <wink>) something related to saving __main__.__spec__.name in the pickle rather than __main__.__name__. Anyone have anything more concrete than that? I'm willing to work this out but only with a more specific goal relative to the pickle module.

    @ericsnowcurrently ericsnowcurrently added the type-feature A feature request or enhancement label Nov 28, 2013
    @ncoghlan
    Copy link
    Contributor

    The specific proposal was to use __spec__.name when __name__ == "__main__"
    to avoid the pickle compatibility issues described in PEP-395 when using
    the -m switch.

    runpy has to be updated first, though.

    @ncoghlan
    Copy link
    Contributor

    bpo-19700 means that runpy now ensures that __main__.__spec__ is set appropriately when __main__ is executed via the import system.

    bpo-19946 means that multiprocessing now ensures that __main__ is configured correctly in child processes to reference a properly initialised "fake main" to allow pickle compatibility with classes and functions defined in __main__ outside "if __name__ == '__main__'" guards.

    The proposal here is that we make the following changes:

    • runpy will ensure that when __main__ is executed via the import system, it will also be aliased in sys.modules as __spec__.name
    • if __main__.__spec__ is set, pickle will use __spec__.name rather than __name__ to pickle classes, functions and methods defined in __main__
    • multiprocessing is updated appropriately to skip creating __mp_main__ in child processes when __main__.__spec__ is set in the parent process

    While I still think this is a reasonable idea, I think it qualifies as a new feature, and hence is better postponed to Python 3.5

    @ncoghlan ncoghlan changed the title Update pickle to PEP 451 Update pickle to take advantage of PEP 451 Dec 17, 2013
    @larryhastings
    Copy link
    Contributor

    So far I agree that this should be postponed to 3.5.

    @ncoghlan
    Copy link
    Contributor

    Just noting that PEP-499 covers adding modules executed with -m to sys.modules under their real name in addition to __main__.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir topic-importlib topic-multiprocessing type-feature A feature request or enhancement
    Projects
    Status: No status
    Status: No status
    Development

    No branches or pull requests

    6 participants