The current execution model documentation in the Language Reference doesn't clearly explain the multiple phases of code execution:

1. Compilation time (statement by statement in the main module and at the interactive prompt, all at once for module import and the exec() and compile() builtins)
2. Definition time for function statements (i.e. when the function statement itself is executed)
3. Execution time for function and generator bodies (i.e. when a function is called and when next() is invoked on a generator)

Isn't 'definition time' vs 'execution time' an artificial distinction?  I'm surprised that the main module is compiled differently than a regular module.  Is that an artifact of the CPython implementation or a part of the language?  If the latter it should certainly be documented. 

What are the concrete differences between 'execution time' and 'defintion time' that are missing from the language specification?

I presume we are discussing Chapter 4. Execution model. I do not see any mention there of the difference between function definition and body execution time. The section of def statements (7.6) has this: "The function definition does not execute the function body; this gets executed only when the function is called. [3]". This is true for lambda expressions also, but nothing is said there. This may contribute to people expecting name resolution of non-parameters in lambda bodies to happen at lambda expression == function definition time rather than later at function call time (what Nick called function execution time).

I think 4.1. Naming and binding should be slightly expanded and renamed 4.1. Name binding and resolution. People need to understand both how and when associations are made and when they are used (as well as what scopes are used). Not understanding this results in the perennial confusion about params with defaults and free names in lambda expressions.

In function headers, param = expr creates a binding when the header is executed, which is when the function object is created. It is stored in the function object. It may optionally be used when the function is called.

In lambda bodies such as in lambda x: x + i, there is no binding of i so i must be bound in a surrounding context either before the function is defined or before the function is called, depending on the surrounding contex, and used when the function is called.

These two facts are somehow not clear enough at present. In any case, my point is the the 'when' part of name binding and use is related to the when of execution.

The relevance of compilation time is more subtle. It is when module, statement, and function code objects are created, including by compile(). If code objects (and compile) are part of the language, then so is compile time. The existence of both interactive statement execution and delayed module execution also has implications for how Python works and how it can and cannot be modified. I am not sure how much to say, though.

Part of the execution model of Python is that, leaving aside i/o, the effect of executing statements is to change user visible name and slot bindings. There are essentially no declarations for changing invisible compiler/interpreter settings.

Most of my thoughts on this topic can be found at http://svn.python.org/view/sandbox/trunk/userref/ODF/ (that was written ~2.5, so some parts of it are a little dated, but the Essential Concepts section is still fairly accurate).

The most relevant part to the current discussion is 1.5:

1.5 Classes and Functions are Defined at Runtime
In static language such as Java or C++, function and class definitions are essentially directives to the language compiler regarding the nature of certain identifiers. While class and function definition statements still have implications for the compilation stage in Python, these definitions are also first class statements that are executed at runtime, just like any other statement. While the code within the function or class definition statement is compiled at compile time, the actual definition of the function or class does not occur until the statement is executed at runtime.
In top level code, this distinction usually doesn't matter, but it has some significant implications when class and function definitions are placed inside a function. Doing so means that a new class or a new function is defined every time the containing function is executed. This means that not only is it possible to have factory functions that create new class instances (as is common in all object-oriented languages), but it is also possible to have factory functions that create new classes or new functions. Another key feature of nested functions is that they employ lexical scoping, allowing nested functions to see identifiers defined in outer scopes.

The other directly relevant part is in section 5:

5.1.3 Evaluating Default Arguments
In static languages such as C, function definitions are instructions to the compiler rather than executable statements. This leads to such languages making distinctions between events that occur at run-time and at compile time. In Python, function definitions are statements in their own right, so run-time events related to functions are further divided into events that occur at definition time and at execution time. Definition time refers to the point where the function is defined by the def statement. Execution time refers to the point where the function is called and the body of the function is executed.
Where these distinctions matter the most is in the evaluation of default arguments, as this occurs only once at the time the function is defined. For immutable default values like numbers or the constant value None, this doesn't have any perceptible effect. For mutable defaults like lists and dictionaries, however, it makes a significant difference, as it means a single instance of the object is created at definition time, and is then shared across all invocations of the function that use the default argument value. This can result in unintended side effects. The typical approach to handling this situation is to use None as the default argument, and include code in the body of the function to create the appropriate default value when the parameter is set to None. The following listing shows the difference between the two mechanisms.
[Grab the ODF file if you want to see the code listing]

My understanding is that the language reference is a purposefully minimalist document that specifies the language (insofar as anything other than the CPython implementation does so).  So while better explanations of the implications of the language design are a good thing, they don't necessarily belong in the language reference.  (I'm not saying they don't, I'm just repeating what the intro says: "this is not a tutorial".)  In particular I am suspicious that statements that begin "in languages such as ..." don't belong in the language reference as it is currently written.

Oh, agreed - the quotes above are from the "Python's User Reference" I wrote several years back, but never got around to converting from ODF and subsequently updating and publishing in a more accessible way.

It was designed to fill the gap that exists in the documentation between the tutorial (which gives usage information, but doesn't really explain how things work) and the language reference (which is a fairly minimalist spec that doesn't explain design intent or rationale very much). The associated publishing deal fell through so I got the copyrights back and donated the whole thing to the PSF.

However, the first step in making it more useful is extracting the text from the ODF files into reStructuredText and that's never made it to the top of my to-do list.

The ideal situation from my point of view would be for someone on doc-sig to help with the conversion process, but I've never had any takers (although I'll grant that the last time I asked was a number of years ago).

You could try the mentors list or Doug Hellman's volunteer group, too.

Hi Nick,

Would you be able to update the link as http://svn.python.org/view/sandbox/trunk/userref/ODF/ isn't valid anymore?  Have things changed even more since you created this issue or are you still interested in pursuing this?  Thanks!

https://github.com/ncoghlan/python-userref/blob/master/ODF/Chapter01_EssentialConcepts.odt is the Chapter 1 userref link now that the PSF SVN server has shut down.

Depending on what you mean by "Pursuing this...":

1. I'd definitely still love to see the content from the user reference converted into a collaborative documentation project and brought up to speed with modern Python - it's just a large enough task that I've shied away from doing it myself.

2. For this ticket itself, I'm thinking this kind of explanatory material might find a better home in the Developer's Guide these days, as there it could also provide pointers to the different parts of CPython that implement each phase.

Cheryl has started the conversion process for the old user reference manuscript [1], so I think it makes sense to mark this as "postponed" for now, until we see how far we can get with this more usage-centric level of explanation as a separate project from the language reference.

[1] https://github.com/ncoghlan/python-userref/tree/master/converted