Author eryksun
Recipients JDLH, eryksun
Date 2017-02-15.07:11:58
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1487142720.43.0.065446739388.issue29561@psf.upfronthosting.co.za>
In-reply-to
Content
For the tokenizer, a blank line is "[a] logical line that contains only spaces, tabs, formfeeds and possibly a comment" [1]. A blank line is normally ignored, except in the REPL an entirely blank line (i.e. no whitespace or comment) is used to end a multi-line statement.

This behavior is coded in Parser/tokenizer.c in tok_get(). After removing leading whitespace to get the indentation level, it decides whether the line should be ignored as blank as follows:

        if (c == '#' || c == '\n') {
            /* Lines with only whitespace and/or comments
               shouldn't affect the indentation and are
               not passed to the parser as NEWLINE tokens,
               except *totally* empty lines in interactive
               mode, which signal the end of a command group. */
            if (col == 0 && c == '\n' && tok->prompt != NULL) {
                blankline = 0; /* Let it through */
            }
            else {
                blankline = 1; /* Ignore completely */
            }
            /* We can't jump back right here since we still
               may need to skip to the end of a comment */
        }

The tokenizer switches to the ps2 prompt unconditionally, even if the first line was ignored as a blank line. One can argue that this behavior is outside of the norm for a shell or REPL. For example, bash doesn't switch to its PS2 prompt after ignoring an initial blank line. On the other hand, the interpreter is correctly conveying that it's still tokenizing the input; it hasn't compiled or executed any code.

[1]: https://docs.python.org/3/reference/lexical_analysis.html#blank-lines
History
Date User Action Args
2017-02-15 07:12:00eryksunsetrecipients: + eryksun, JDLH
2017-02-15 07:12:00eryksunsetmessageid: <1487142720.43.0.065446739388.issue29561@psf.upfronthosting.co.za>
2017-02-15 07:12:00eryksunlinkissue29561 messages
2017-02-15 07:11:58eryksuncreate