classification
Title: Document the possible values for __main__.__spec__
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: 19700 Superseder:
Assigned To: Nosy List: Arfrever, brett.cannon, eric.snow, ncoghlan, python-dev
Priority: normal Keywords: patch

Created on 2013-11-22 16:25 by brett.cannon, last changed 2014-03-29 05:41 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
issue19697-basic.diff eric.snow, 2014-03-28 21:29 basic initial additions review
Messages (17)
msg203796 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 16:25
So that e.g. __main__.__spec__.name is the actual name of the module being executed.
msg203945 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-23 01:23
I've already started on this somewhat.  However, I'm going to close out more of the other PEP 451 issues before going any further.

Nick: when do you think things will settle down that you could field a question or two and some reviews relative to the __main__ changes.  I keep wondering if I'll have inadvertently implemented good chunks of PEP 395 by the time I'm done here. <wink>
msg204030 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-23 14:27
Yes, PEP 451 addresses a few of the PEP 395 issues. Between this and PEP
420, I actually plan to withdraw that PEP as out of date.

All my pre-beta commits are done, so ask away :)
msg207267 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-04 07:09
Here's an outline of how I see __main__.__spec__ playing out relative to the various cmdline interfaces.

== == == === ==== ======== ======== ======== ======== ============
-  -c -m dir file name     loader   origin   cached   has_location
== == == === ==== ======== ======== ======== ======== ============
                  __main__ builtin  -        None     False
X                 __main__ builtin  -        None     False
   X              __main__ builtin  -c       None     False
      X           (finder) (finder) (finder) (finder) (finder)
         X        __main__ <loader> path     None     True
             X    __main__ <loader> path     None     True
== == == === ==== ======== ======== ======== ======== ============

Note: __main__.__spec__ in the -m case is addressed in issue #19700.

Thoughts?

See:

[1] http://docs.python.org/3.4/using/cmdline.html
[2] Python/pythonrun.c
[3] Modules/main.c
msg207268 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-04 07:10
For the record, normal start-up happens like this (simplified):

1. prep for and create/initialize interpreter
2. exec the site module in the __main__ namespace (unless -S)
3. do the interface-specific stuff

Note: exec of the site module does not impact the spec.

In the case where -i/PYTHONINSPECT is issued (or implied):

1. ...
2. ...
3. exec the PYTHONSTARTUP script (if set and it exists)
4. do the interface-specific stuff
5. start the REPL

Note: the -i case does not impact the spec, nor does the exec of any PYTHONSTARTUP script.


See:

[1] http://docs.python.org/3.4/using/cmdline.html
[2] Python/pythonrun.c
[3] Modules/main.c
[4] PEP 432
msg207269 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-04 07:14
Food for thought:

We could (for 3.5) add an importer just for __main__ that gives us the appropriate spec and loads __main__ accordingly.  Such a loader could even incorporate exec of the site module (and any PYTHONSTARTUP script).
msg207271 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-04 07:45
You'd also need to update the new multiprocessing code - it currently expects "__main__.__spec__ == None" for all the run-from-a-path-or-stdin cases.

The -m switch and running __main__ from a supplied sys.path entry (the "dir" entry in your table) are both already handled by the runpy changes in issue 19700.

The remaining cases where __main__.__spec__ is currently None:

- interactive prompt
- -c switch
- running from stdin
- running directly from a source or bytecode file

To be honest, I'm not sure it actually makes sense to try to manufacture a pseudo-spec for those cases. A main script may not be importable as a module (e.g. a hyphen in its name, or no .py suffix), and you *definitely* can't import a file that doesn't exist on disk (REPL, stdin, -c).
msg207272 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-01-04 08:09
I'm fine with leaving __spec__ as None for those remaining cases.  It definitely simplifies this ticket. :)  Do you see any reason to not close this one at this point?
msg207273 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-01-04 08:20
I think we need to document this somewhere. Not exactly sure where though - a new subsection in the import reference, perhaps?
msg215078 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-28 21:29
Here's a basic patch.  It introduces __main__ in import.rst and then identifies the cases where __spec__ is set to None or otherwise.

There are a few :ref: links to labels on other pages that I had trouble with.  I'll address those when I put up a new draft after feedback.  Any suggestions on making them work would be helpful. :)
msg215093 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-28 23:31
Looks like a reasonable general structure to me.

For label references, you don't want to include the leading underscore - that's part of the label syntax, not the label itself.
msg215098 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-28 23:46
That did the trick, Nick.  I swear there's always something I forget about ReST. :)

Otherwise do you have any objections to the patch?  I imagine there's more we could put in the section (for language spec purposes), but I figure what I have there is good enough for the moment.
msg215099 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-28 23:48
"is some cases" is a typo, but otherwise, yeah, may as well commit this so the website gets updated while we decide what else (if anything) we want to cover.
msg215100 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-29 00:11
New changeset 21d8222b667f by Eric Snow in branch 'default':
Issue #19697: Document cases where __main__.__spec__ is None.
http://hg.python.org/cpython/rev/21d8222b667f
msg215101 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-29 00:30
New changeset 4972475ae2e7 by Eric Snow in branch '3.4':
Issue #19697: Document cases where __main__.__spec__ is None.
http://hg.python.org/cpython/rev/4972475ae2e7
msg215102 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-03-29 00:32
Wish I had done that in the opposite direction.  Anyway, thanks Nick.
msg215114 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-29 05:41
New changeset 0427c2ff653d by Nick Coghlan in branch '3.4':
Issue #19697: document more __main__.__spec__ quirks
http://hg.python.org/cpython/rev/0427c2ff653d

New changeset a90254be2da2 by Nick Coghlan in branch 'default':
Issue #19697: merge docs from 3.4
http://hg.python.org/cpython/rev/a90254be2da2
History
Date User Action Args
2014-03-29 05:41:34python-devsetmessages: + msg215114
2014-03-29 00:32:06eric.snowsetstatus: open -> closed
versions: + Python 3.5
messages: + msg215102

resolution: fixed
stage: test needed -> resolved
2014-03-29 00:30:05python-devsetmessages: + msg215101
2014-03-29 00:11:54python-devsetnosy: + python-dev
messages: + msg215100
2014-03-28 23:48:37ncoghlansetmessages: + msg215099
2014-03-28 23:46:48eric.snowsetmessages: + msg215098
2014-03-28 23:31:14ncoghlansetmessages: + msg215093
2014-03-28 21:29:30eric.snowsetfiles: + issue19697-basic.diff
keywords: + patch
messages: + msg215078
2014-01-04 08:20:22ncoghlansetmessages: + msg207273
title: refactor pythonrun.c to make use of specs (__main__.__spec__) -> Document the possible values for __main__.__spec__
2014-01-04 08:09:52eric.snowsetmessages: + msg207272
2014-01-04 07:45:39ncoghlansetmessages: + msg207271
2014-01-04 07:14:23eric.snowsetmessages: + msg207269
2014-01-04 07:10:23eric.snowsetmessages: + msg207268
2014-01-04 07:09:10eric.snowsetmessages: + msg207267
2013-12-13 19:53:20brett.cannonlinkissue19978 dependencies
2013-12-13 19:52:36brett.cannonunlinkissue19701 dependencies
2013-12-06 05:48:17eric.snowlinkissue19701 dependencies
2013-12-06 05:47:53eric.snowsetdependencies: + Update runpy for PEP 451
2013-11-23 19:50:11Arfreversetnosy: + Arfrever
2013-11-23 14:27:21ncoghlansetmessages: + msg204030
2013-11-23 01:23:00eric.snowsetmessages: + msg203945
2013-11-22 18:43:13eric.snowlinkissue19709 superseder
2013-11-22 16:26:00brett.cannonlinkissue18864 dependencies
2013-11-22 16:25:46brett.cannoncreate