classification
Title: Document that compile(code, 'exec') has different behavior in 3.7+
Type: Stage: resolved
Components: Documentation, Interpreter Core Versions: Python 3.8, Python 3.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, inada.naoki, mbussonn
Priority: normal Keywords: patch

Created on 2018-05-13 00:57 by mbussonn, last changed 2018-06-05 18:28 by mbussonn. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6973 closed mbussonn, 2018-05-18 17:41
Messages (5)
msg316441 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2018-05-13 00:57
In recent Python the following 

>>> from ast import PyCF_ONLY_AST
>>> compile("'a'", 'whatever', 'exec', PyCF_ONLY_AST).body

In 3.6 it return

   [<_ast.Expr at 0x10b7441d0>] # that contail Str('a')

While on master:

   []

This is inconveninent for alternative repl like IPython, where basically if the user is entering a single string, the result is nothing in Python 3.7+, while it _does_ return something on earlier Python [1]. 

The documentation of `compile` says:

>  ... it can be 'exec' if source consists of a sequence of statements,

Which is not technically true any more as the first statement, if a string, will be removed. 

What's happening here is that since Python 3.7 if the _first_ statement is actually an expression containing a lonely string it is assign to the module docstring. So that's basically assuming you are parsing a module, and that the docstring make sens in this context, while in a REPL you are parsing a sucesssion of statements, in which case there is no need for a docstring that make no sens in this context.


This is _usually_ not an issue, unless this lonely statement is also the last, and what the user wants to execute in a REPL, in which case it has no side effect.

I don't have any objection to the new behavior, though I was wondering if this kind of side effect was anticipated. 

If that affect IPython, it will likely effect other alternative REPLs.

Thus, I believe it would be good to have this at least documented a tiny bit better and added in what's new, and potentially clarified in the `exec` docs.

I could argue that now the  `exec` name may be a tiny bit unsuitable for the new behavior, and would love if this could be optional.

1: https://github.com/ipython/ipython/issues/11133#issuecomment-388591332
msg316490 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2018-05-14 03:36
ref #32911

And it's documented already.
https://docs.python.org/3.7/whatsnew/3.7.html#changes-in-the-python-api
msg317036 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2018-05-18 16:37
Any comments?
Would you close this issue?
msg317037 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2018-05-18 17:02
Fair enough that what's new include things about Module


> The first statement in their body is not considered as a docstring anymore.

Note that this sentence read backward to me. I understand what is meant because I know the new behavior. It might be good to clarify. potentially:

> The first statement in the `body` attribute of should not be considered the docstring of the module anymore, the `docstring` attribute  is reserved for that.


Though the documentation of `compile()` does not say that `compile(...,'exec')` compile a module. It says:

> The mode argument specifies what kind of code must be compiled;
> it can be 'exec' if source consists of a sequence of statements

Which now is incorrect. I was expecting `compile(..., 'exec')` to return a Module with `None` or empty string as the docstring attribute – which is also a perfectly reasonable request.

I think that `compile`  documentation should be changed to reflect what it does. 


Or (but I see why this is un-reasonable) split add the `mode='module'` that has new behavior, while `exec` does not.
msg318779 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2018-06-05 18:28
Closing as the ast changes have been reverted.
History
Date User Action Args
2018-06-05 18:28:22mbussonnsetstatus: open -> closed

messages: + msg318779
stage: patch review -> resolved
2018-05-18 17:41:17mbussonnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request6628
2018-05-18 17:02:04mbussonnsetmessages: + msg317037
2018-05-18 16:37:58inada.naokisetmessages: + msg317036
2018-05-14 03:36:43inada.naokisetnosy: + inada.naoki
messages: + msg316490
2018-05-13 00:57:30mbussonncreate