This issue is to track the implementation of PEP 499 which we hope to get into the upcoming 3.8 release. I've made it because cpython PRs want a bpo number in the subject line.

I'll link in the proposed PR once I've filed off a few grammar issues I've just noticed (documentation English grammar).

I've withdrawn the PR; I hadn't run the full test suite and there are things to fix. - Cameron

New PR 12490 attached with a fix for test_pdb. More extensive comments are in the leading comment on the PR itself. - Cameron

[Belatedly updating this issue with the current status as of March]

Cameron's implementation generally looks good, but there are couple of compatibility/migration questions that we need to consider, as spelled out in the PEP update that added me as BDFL-Delegate: https://github.com/python/peps/pull/946/files

* We need a generic porting guide entry to handle projects that turn out to have been relying on their name *not* being bound in sys.modules. For example, adding this preamble:

    if __name__ == "__main__":
        # To prevent inadvertent double imports, the -m
        # switch in Python 3.9+ defaults to aliasing __main__
        # under the executed module's import name. We actually
        # want the double import, so remove the alias if it exists
        import sys
        _main_module = sys.modules.get(__name__)
        _spec_module = sys.modules.get(__spec__.name)
        if _main_module is _spec_module:
            sys.modules.pop(__spec__.name)

We'd test the above snippet by adding it to the `pdb` module (and reverting the other compatibility changes to that module)

* We need to understand the implications for pickle compatibility, and provide a porting guide snippet, similar to the one above for explicitly requesting the double-import behaviour. For example:

    if __name__ == "__main__":
        # To prevent inadvertent double imports, the -m
        # switch in Python 3.9+ defaults to aliasing __main__
        # under the executed module's import name. We need
        # pickle to use the real module name for objects from
        # __main__ though, so we set the import name here
        _running_as_main = True
        __name__ = __spec__.name

I want to start with an apology. I have become a little swamped by work and didn't let anyone know I've made little time for this since March.

To my naive eye Nick's snippet looks like it would work for pdb; I became a little stalled there trying to understand how pdb managed the main module.

I totally lack the expertise to address pickle; I've never used it and the long history of feature updates sugests to me that I cannot get a deep enough understanding in a meaningful time.

Just a remark about the preamble comment: it reads to me as though PEP499 is a misfeature. Possibly change:

  We actually want the double import, so remove the alias if it exists.

to:

  This module is unusual, and actually wants the double import.
  Therefore we remove the __spec__.name alias if it exists
  so that a subsequent import by name will make a distinct instance.

Verbose, but at least I know the intent.

FWIW, I have some feedback on the PEP.  (See msg357448.)  Can we discuss here or should I open a mailing list thread?

On 25Nov2019 17:38, Python Bug Reports <report@bugs.python.org> wrote:
>Eric Snow <ericsnowcurrently@gmail.com> added the comment:
>
>FWIW, I have some feedback on the PEP.  (See msg357448.)  Can we discuss here or should I open a mailing list thread?

Let's discuss it here unless it looks like we need wider input. This is 
related to issue37292 (_xxsubinterpreters: Can't unpickle objects 
defined in __main__), yes?

With PEP499, "python -m foo" should have __name__=='__main__' and 
__spec__.name=='foo'. Were you thinking the __module__ should come from 
__spec__.name (and that the PEP should make that clear)?

Exactly. :)

I'd expect PEP 499 to specify changing __module__ of classes and functions from __main__ to the module name (__spec__.name).  This aligns closely with the whole point of the PEP. :)  As a bonus, it will simplify things for pickling (which doesn't inherently deal well with __main__).

Leaving the relationship between pickle and __name__ alone wasn't an oversight, as folks already rely on that to gracefully transition from single-file modules to multi-file packages without breaking pickle compatibility in either direction. The trick is to do a "from ._submodule import *" in the package's __init__.py (exposing all the names), and then a "__name__ = __package__" in the submodule itself.

Thus the second snippet above, as a way to port code that was specifically relying on the double import to provide pickle compatibility without risking introducing other unintended incompatibility problems.