This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author terry.reedy
Recipients Esa.Peuha, Mark.Shannon, RJ722, Rosuav, cheryl.sabella, lys.nikolaou, mbussonn, ncoghlan, pablogsal, r.david.murray, terry.reedy
Date 2020-07-01.04:13:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1593576810.13.0.162916716917.issue19335@roundup.psfhosted.org>
In-reply-to
Content
I reread Nick's comment "the C level loop simply blocks on stdin, waiting until the parser spits out a complete AST."  I interpret that now as meaning that the REPL compiles user code only once per statement, which IDLE and code.InteractiveInterpreter compile a statement once per line, which codeop expands to 3 compiles in an attempt to determine whether more is needed.  This can all be considered to be a hack, so adding 1 more does not bother me now.

(Side note: if the first compile succeeds in producing a code object, it is unconditionally returned after the additional compiles, so they seem like a waste.)

I looked at a git blame listing for codeop.  It was extracted from code by Guido in 1998 so that Jython (JPython) could replace compile_command.  It was not revised after the addition of nonlocal broke it.

I traced the flow of calls from when a user hits <Enter> in the entry area to the call of compile, and of the returns back.  The attached idle-shell-compile.txt records what I found.  Of particular interest to me:

1. IDLE strips trailing ' 's, '\t's, and 1 '\n' from user input before sending it to be compiled. (This goes back to the original IDLE commit in 2000.) So there is a trailing '\n' only when the user has hit '<Enter>' twice to complete a compound statement.  It would solve this issue *for IDLE* to only print the nonlocal error message when there is a trailing '\n'.  I am willing to give up the special casing needed to get 'def a():\n  nonlocal c' (no double <Enter>, no trailing '\n') to raise immediately.

Nothing in code says that it expects source to be stripped as IDLE does.  The reason for the latter might just be to get rid of the 'smart indent' that will precede the first Enter, and possibly anything the user adds before the second.  

2. compile is passed the undocumented flag PyCF_DONT_IMPLY_DEDENT = 0x200.  Can anyone explain the full intent?  I found this difference.

>>> compile("def f():\n  nonlocal c", '', 'single')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 2
SyntaxError: no binding for nonlocal 'c' found

# Same code, flag added, different message.
>>> compile("def f():\n  nonlocal c", '', 'single', 0x200)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 2
    nonlocal c
             ^
SyntaxError: unexpected EOF while parsing

# Add '\n' and message is same as without flag.
>>> compile("def f():\n  nonlocal c\n", '', 'single', 0x200)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 2
SyntaxError: no binding for nonlocal 'c' found

To use the message difference, the first compile-with-flag message would have to be saved for comparison, and the difference would only appear if source had no trailing '\n'.  We could make that be true, or we could wait until another user of codeop complains.
History
Date User Action Args
2020-07-01 04:13:30terry.reedysetrecipients: + terry.reedy, ncoghlan, r.david.murray, Mark.Shannon, Rosuav, Esa.Peuha, mbussonn, cheryl.sabella, lys.nikolaou, pablogsal, RJ722
2020-07-01 04:13:30terry.reedysetmessageid: <1593576810.13.0.162916716917.issue19335@roundup.psfhosted.org>
2020-07-01 04:13:30terry.reedylinkissue19335 messages
2020-07-01 04:13:29terry.reedycreate