classification
Title: REPL shows continuation prompt (...) when comment or space entered
Type: Stage: patch review
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Phaqui, gvanrossum, terry.reedy
Priority: low Keywords: patch

Created on 2019-11-03 16:15 by gvanrossum, last changed 2019-11-30 21:43 by BTaskaya.

Pull Requests
URL Status Linked Edit
PR 17421 open BTaskaya, 2019-11-30 21:43
Messages (7)
msg355903 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-03 16:15
This has always bothered me, and it shouldn't be necessary.

This session:

>>> #foo
... 
>>> 

should really have been

>>> #foo
>>>

It's confusing that the REPL prompt switches to "..." here, for no good reason. It should just treat the line as empty.

Ditto if you enter a space (there's an invisible space after the first prompt):

>>>  
... 
>>>
msg356205 - (view) Author: Anders Lorentsen (Phaqui) * Date: 2019-11-07 17:52
As a person without much experience, it sounded like a simple enough task, but having dug a bit, I found it quite complicated. It seems to me that the interpreter loop (in the standard REPL, that you get when you start ./python, blocks for input somewhere inside a massive function called 'parsetok' (in Parser/parsetok.c). Now, I could maybe investigate further, to have it return to the interpreter loop if it reads a comment (or empty line), but I'm afraid to mess up something.

From my understanding, there aren't that many other choices, because parsetok() doesn't return before you finish the statement (in other words, it does not return if you type a comment line or a blank line - it instead waits for more input, as indicated by the '... ').

Am I way off in concluding that this would be a change to the parser?
msg356214 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-07 22:46
Yes, that's likely where the change should be made.

I think if the *first* token encountered is either NL or COMMENT the parse should be abandoned by the tokenizer.
msg356271 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-11-08 22:55
Entering 'pass' or a completely blank line results in a new primary prompt, at least on Windows. The Windows REPL otherwise prints ... even for effectively blank lines.  IDLE usually prints a new prompt for effectively blank lines.

>>> 
>>> #a
>>> # a
>>>  #a
>>>

I agree that these look better.  This behavior comes from code.InteractiveInterpreter and ultimately codeop.

def _maybe_compile(compiler, source, filename, symbol):
    # Check for source consisting of only blank lines and comments
    for line in source.split("\n"):
        line = line.strip()
        if line and line[0] != '#':
            break               # Leave it alone
    else:
        if symbol != "eval":
            source = "pass"     # Replace it with a 'pass' statement

As noted above, 'pass\n' is treated the same as '\n'

The first line above originally had a space, but IDLE appears to strip trailing whitespace also, even outside of comments.  (For an ending '\ ', this prevents SyntaxError, but maybe this is a bad lesson for beginners.)  However, I did find a case with an unnecessary continuation line.

>>>  # a
 
>>> 

This puzzles me, as it should be treated exactly the same as without the space after '#'.  ast.dump(ast.parse(' # a\n', '', 'single')) gives the same result, 'Module(body=[], type_ignores=[])', as without.
msg356319 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-10 05:53
Regarding the IDLE mystery, *if* there's a difference between how it treats " # a" and "# a", this must be due to some part of the code that's invoked before _maybe_compile() is called, right?

But that's immaterial to this bug -- I'm only complaining about the "builtin" REPL.
msg356346 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-11-11 03:24
Fix corner case bugs in IDLE would definitely be a separate issue.  But is the 'fix' in _maybe_compile at all applicable to the REPL?  Or might a parser change REPL fix make the code in _maybe_compile unneeded?
msg356348 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-11 04:11
> But is the 'fix' in _maybe_compile at all applicable to the REPL?  Or might a parser change REPL fix make the code in _maybe_compile unneeded?

I don't know.  Most of the contortions in code.py codeop.py are meant to emulate what the parser does without help in the REPL: keep asking for input until one of the following happens:

- a syntax error is found;

- a simple statement is parsed till completion;

- an empty line is found at a point where a compound statement is potentially complete.

The bug here is that apparently the above conditions aren't quite enough, and it seems we need to add:

- a line that contains just whitespace and/or a comment is seen before anything else.

The reason that IDLE has to reimplement similar logic is that the parser (actually, the tokenizer) isn't written as a coroutine to which you send lines you read -- it's written as a blocking function that you pass a file and the function will attempt to read lines from the file.  The function returns a parse tree or raise an error.  That model doesn't work in IDLE, which needs to stay reactive while the shell window is in the middle of a statement.

I think we'll find that the bug is *very* old.  IIRC the initial releases of Python didn't have the rule that indentation is ignored between matching parentheses/brackets/braces.  In those days the tokenizer also didn't gracefully skip blank lines, or lines with comments that weren't aligned with the current indentation level.  So then checking for empty lines was good enough.
History
Date User Action Args
2019-11-30 21:43:10BTaskayasetkeywords: + patch
stage: patch review
pull_requests: + pull_request16902
2019-11-11 04:11:11gvanrossumsetmessages: + msg356348
2019-11-11 03:24:48terry.reedysetmessages: + msg356346
2019-11-10 05:53:43gvanrossumsetmessages: + msg356319
2019-11-08 22:55:09terry.reedysetnosy: + terry.reedy
messages: + msg356271
2019-11-07 22:46:49gvanrossumsetmessages: + msg356214
2019-11-07 17:52:55Phaquisetnosy: + Phaqui
messages: + msg356205
2019-11-03 16:15:35gvanrossumcreate