classification
Title: Better specify number of nested scopes
Type: enhancement Stage: patch review
Components: Documentation Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Mariatta Nosy List: Hibou57, Mariatta, Roscoe R. Higgins, cheryl.sabella, docs@python, eryksun, ezio.melotti, miss-islington, nanjekyejoannah, terry.reedy, veky
Priority: normal Keywords: patch

Created on 2016-01-26 02:16 by Roscoe R. Higgins, last changed 2020-11-04 21:36 by brett.cannon.

Pull Requests
URL Status Linked Edit
PR 21324 merged nanjekyejoannah, 2020-07-05 02:30
PR 21341 merged miss-islington, 2020-07-06 01:42
PR 21342 merged miss-islington, 2020-07-06 01:42
Messages (20)
msg258941 - (view) Author: Roscoe R.Higgins (Roscoe R. Higgins) Date: 2016-01-26 02:16
In chapter 9. Classes of the Python3.5 documentation it says: 

"At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:",

followed by a list containing 4 items.
Further down a middle scope is mentioned (although mentioned by name). This was confusing for a while.
msg258964 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-26 17:37
https://docs.python.org/3/tutorial/classes.html for the docs that Roscoe is talking about.

So the sentence is technically correct, it just takes careful reading to grasp what's being said. There are "at least three nested scopes", but there can be *up to* four scopes. Since "the scopes of any enclosing functions" is not necessarily true for all code scopes, you end up with at least three, but possibly four scopes.

Obviously the wording could be clearer, so if you want to sign the CLA, Roscoe, and propose a rewording that would be appreciated!
msg258978 - (view) Author: Yannick Duchêne (Hibou57) Date: 2016-01-26 22:07
Better at least two, if I'm not wrong: the innermost scope may be the module's scope. So there is always at least the module scope and the built‑ins scope, at least two up to four.

(if I have not overlooked something)
msg258980 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-26 22:21
It depends on how you want to view things as to whether you can claim there are two or three scopes for module-level code. While it's true that the local scope operates just like global scope, to Python it's more like local == global rather than the local scope simply doesn't exist (hence why `locals()` never throws an exception saying the scope doesn't exist but instead is the same as `globals()`).

The "three scope" phrasing also predates nested scopes from back when Python had its LGB scoping rules: Local-Global-Builtin. Back then we just said Python had three scopes and left it at that since it was basically always true and didn't confuse anyone.

At this point I'm fine with just removing the number from the sentence and saying something like "At any time during execution, there are various nested scopes whose namespaces are directly accessible:".
msg259228 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-01-30 01:39
Would 'three or more' be any clearer than 'at least three'?  They mean the same, but the first seems better to me in this context.

The real problem with this section are a) the use of Guido's first person 'I' and b) statements that were not changed when nested scope were added, but should have been.

"If a name is declared global, then all references and assignments go directly to the middle scope containing the module’s global names."

The global scope is no longer the middle scope.  Roscoe pointed at this.  With that removed, the sentence says that if a name is declared global, assignments go to global scope.  This would be more meaningful if prefixed by a revised version of the following, which is several paragraphs down.

"A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope."

The special quirk part should go; 'global' would now have to be 'global or nonlocal', but I now think the following, preceeding the revised 'global' sentence above, would be better.

"By default, assignments to names always go into the innermost, local, namespace."

In other words, I think the sentence Roscoe flagged is the least of the problems with this section.
msg340000 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2019-04-11 22:41
Assigning to @Mariatta for the sprints.
msg373022 - (view) Author: Vedran Čačić (veky) * Date: 2020-07-05 09:27
Just to point out: it can be more than four, right? If you have a function aa within a function bb within a function cc, then bb's and cc's scope are different, even though functionally equivalent from the perspective of aa.
msg373048 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-05 21:13
At any one time, the language and interpreter sees the local scopes of enclosing functions as collectively 'nonlocal' for the purpose of accessing and rebinding.  If there are bindings of a name 'x' in multiple enclosing local scopes, the binding for 'x' in the synthesized 'nonlocal' is the innermost one.
msg373049 - (view) Author: Vedran Čačić (veky) * Date: 2020-07-05 21:34
How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."? Any ChainMap presents itself as a dictionary to the outside. That doesn't mean it doesn't have an internal structure.

Yes, nonlocal picks the innermost one because of its semantics, but it still needs to search through the scopes in order to know which one to pick, right?
msg373055 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-06 00:52
Vedran: "How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."?"

For access, one may bypass the default staged access by specifying a specific namespace.  For rebinding elsewhere than in locals, one must specify one of the others.  So the 4 scopes are each visible.  However, the internal 'structure' of nonlocals is intentionally not visible or accessible.  (The proposal to make them somehow accessible was considered and rejected.)
---

The PR replaces 'at least three' with 'various'.  This is needlessly vague. There are always 3 namespaces and sometimes a 4th, so I think 'at least three' should be replaced with the specific '3 or 4'.

I rechecked two special cases.  Exception clauses with 'as name' bind name in the current local namespace and auto delete the binding when done.  Comprehensions bind the name and iterator of the 'for' clause in a temporary new local namespace, making the existing namespace temporarily part of a nonlocal namespace, which may or may not have been present before.  '3 or 4' still applies in both cases.
msg373056 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2020-07-06 00:58
PR updated with your change. @Terry
msg373058 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2020-07-06 01:42
New changeset 9ed3cd8ba052b395ab50692bb65988b065d68e27 by Joannah Nanjekye in branch 'master':
bpo-26205: Specify the number of nested scopes (GH-21324)
https://github.com/python/cpython/commit/9ed3cd8ba052b395ab50692bb65988b065d68e27
msg373059 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2020-07-06 01:45
Merged, this can be closed if there is consesus.
msg373061 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2020-07-06 01:53
New changeset 7ceb3e3ffc8ee00551df2245544eb60f7debf3af by Miss Islington (bot) in branch '3.8':
bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21342)
https://github.com/python/cpython/commit/7ceb3e3ffc8ee00551df2245544eb60f7debf3af
msg373062 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-06 01:59
I wait for backports to be merged before closing.  Someone has to approve them or manually merge them before that happens, so leaving issues open until all merges are done prevents totally forgetting about them.

For doc issues, the patch, if not drastically wrong, becomes the new status quo.  So I think any further proposed change should normally be a new issue.  So once the backports *are* done, you can close.

For code issues, 'drastically wrong' usually means that the patch failed on one of the buildbots, or fairly soon somewhere else, and needs either reversion or a hot fix.
msg373063 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2020-07-06 02:06
New changeset 3f4a9fd912fc6d4f5ee2b49bfef979cc7d457848 by Miss Islington (bot) in branch '3.9':
bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21341)
https://github.com/python/cpython/commit/3f4a9fd912fc6d4f5ee2b49bfef979cc7d457848
msg373068 - (view) Author: Vedran Čačić (veky) * Date: 2020-07-06 05:26
Well, then "what name is in which namespace" is relative to which function we're considering. In my example above, imagine aa has x declared, bb has y, and cc has z. Then y and z are in the same namespace when we look from the perspective of aa, but they are not in the same namespace from the perspective of bb. Even worse, cc sees z but doesn't see y. How can they be in the same namespace then?

I always thought about namespaces as mappings from names to objects, independent of perspective. Whether two names are in the same namespace, should be a question with an objective answer.
msg373085 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-06 09:49
Since you ask, here is a extended summary of namespaces. There is one built-in namespace.  There is one global namespace for each module, which is also the local namespace for top level code in that module.  There is one local namespace for each class and function.  There is one nonlocal namespace for each nested function.  "At any time during execution" (the beginning of the doc sentence targeted by this issue), there are 3 or maybe 4 namespaces accessible with simple undotted names. 

Module global and class local namespaces, including nested classes, are accessible from without the object with dotted names.  There are rules for class and method code about access to superclass namespaces.  Function locals are not necessarily accessible with standard python code, but at least in cpython, local names can be accessed via code objects.  Default parameter values and current nonlocal values, when set, can be accessed via function objects.  Dotted names are discussed elsewhere in the tutorial, while function and code object introspection is out of scope there.  (And off topic for this issue ;-).
msg373087 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-06 10:03
Besides the sentence now revised, the initial post referenced confusion with 'middle scope' in "If a name is declared global, then all references and assignments go directly to the middle scope containing the module's global names."

This has not been directly discussed that I could find, but it seems inappropriate both at toplevel, where globals = locals, and in nested functions, where nonlocals are also 'middle'.  Maybe replace "the middle scope containing the module's global names" with "the module's global namespace".  The rest of this paragraph and the next could be reviewed.
msg373125 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-07-06 14:42
> There is one local namespace for each class and function.  There is one
> nonlocal namespace for each nested function.

In the first sentence, replace "class" with "class statement" and "function" with "function call". The second sentence could use "nested-function call", but maybe rewrite it as, "There is one nonlocal namespace when a nested function is called."
History
Date User Action Args
2020-11-04 21:36:36brett.cannonsetnosy: - brett.cannon
2020-07-06 14:42:17eryksunsetnosy: + eryksun
messages: + msg373125
2020-07-06 10:03:00terry.reedysetmessages: + msg373087
2020-07-06 09:49:06terry.reedysetmessages: + msg373085
2020-07-06 05:26:37vekysetmessages: + msg373068
2020-07-06 02:06:19nanjekyejoannahsetmessages: + msg373063
2020-07-06 01:59:02terry.reedysetmessages: + msg373062
versions: + Python 3.8, Python 3.9
2020-07-06 01:53:43nanjekyejoannahsetmessages: + msg373061
2020-07-06 01:45:24nanjekyejoannahsetmessages: + msg373059
2020-07-06 01:42:55miss-islingtonsetpull_requests: + pull_request20490
2020-07-06 01:42:48miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20489
2020-07-06 01:42:30nanjekyejoannahsetmessages: + msg373058
2020-07-06 01:13:26terry.reedysettitle: Inconsistency concerning nested scopes -> Better specify number of nested scopes
2020-07-06 00:58:53nanjekyejoannahsetmessages: + msg373056
2020-07-06 00:52:41terry.reedysetmessages: + msg373055
versions: + Python 3.10, - Python 3.7, Python 3.8
2020-07-05 21:34:31vekysetmessages: + msg373049
2020-07-05 21:13:15terry.reedysetmessages: + msg373048
2020-07-05 09:27:44vekysetnosy: + veky
messages: + msg373022
2020-07-05 02:30:28nanjekyejoannahsetkeywords: + patch
nosy: + nanjekyejoannah

pull_requests: + pull_request20474
stage: needs patch -> patch review
2019-04-11 22:41:55cheryl.sabellasetversions: + Python 3.7, Python 3.8, - Python 2.7, Python 3.5, Python 3.6
nosy: + Mariatta, cheryl.sabella

messages: + msg340000

assignee: docs@python -> Mariatta
2016-03-12 09:34:15ezio.melottisetnosy: + ezio.melotti
type: enhancement
2016-01-30 01:39:40terry.reedysetnosy: + terry.reedy
messages: + msg259228
2016-01-26 22:22:08brett.cannonsetstage: needs patch
type: behavior -> (no value)
versions: + Python 2.7, Python 3.6
2016-01-26 22:21:50brett.cannonsetmessages: + msg258980
2016-01-26 22:07:13Hibou57setnosy: + Hibou57
messages: + msg258978
2016-01-26 17:37:16brett.cannonsetnosy: + brett.cannon
messages: + msg258964
2016-01-26 02:16:27Roscoe R. Higginscreate