New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New parser considers empty line following a backslash to be a syntax error, old parser didn't #85024
Comments
While debugging issues with the black test suite in Python 3.9, I found one which black upstream says is a Cpython issue, so I'm filing it here. Reproduction is very easy. Just use this four-line tester: print("hello, world")
\
print("hello, world 2")
with that saved as `test.py`, check the results:
The reason black has this test (well, a similar test - in black's test, the file *starts* with the backslash then the empty line, but the result is the same) is covered in psf/black#922 and psf/black#948 . |
This is limited to cases where the line continuation character is on an otherwise empty line. For example this works correctly: $ cat t.py
print("hello world")
print("hello world 2") \ print("hello world 3")
$ ./python.exe t.py
hello world
hello world 2
hello world 3 The actual problem is at the tokenizer level, where a line with only a continuation character is not considered an empty line and thus two NEWLINE tokens get emitted, one after the other. The old parser was somehow working around this, probably by having this in the grammar: file_input: (NEWLINE | stmt)* ENDMARKER The PEG parser OTOH does not allow this. The question now is, is it reasonable to change the tokenizer to consider a lone backslash an empty line? Do you also consider this a bug? Or should we change the new parser? |
Sure looks like a tokenizer issue to me. For example this is broken in both versions: pass pass It complains about an unexpected indent, but it should really be considered a blank line broken in two -- a backslash is supposed to just erase itself and the following newline. https://docs.python.org/3/reference/lexical_analysis.html#explicit-line-joining |
I'm not the best person to ask what I'd "consider" to be a bug or not, to be honest. I'm just a Fedora packaging guy trying to make our packages build with Python 3.9 :) If this is still an important question, I'd suggest asking the folks from the Black issue and PR I linked to, that's the "real world" case if any. |
To be clear, I consider it a bug. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: