When I have a normal syntax error in a file, Python reports the filename in the exception output:

$ cat syntax_error.py 
0x=5

$ python3.8 syntax_error.py 
  File "syntax_error.py", line 1
    0x=5
     ^
SyntaxError: invalid hexadecimal literal


But if the syntax error is inside an f-string, Python reports 'File "<fstring>"' instead of the actual filename in the exception output.

$ cat syntax_error_in_fstring.py 
f'This is a syntax error: {0x=5}'

$ python3.8 syntax_error_in_fstring.py 
  File "<fstring>", line 1
SyntaxError: invalid hexadecimal literal

Additionally, the output in the 2nd example does not contain the helpful text printing the context and location of the code containing the syntax error.

This is actually specified behavior in PEP 498
 
https://www.python.org/dev/peps/pep-0498/#expression-evaluation

> Expressions are parsed with the equivalent of ast.parse('(' + expression + ')', '<fstring>', 'eval')

Although Batuhan is correct maybe we can consider trying to improve the debugging experience off-strings due to the fact that these errors in a big project may be difficult to find. Although you have always the back-trace on the other side.

I agree that although the PEP says that's how it behaves, that shouldn't be prescriptive of the error message. Clearly we can be more helpful here.

I have a large, elaborate re-write of the error generating code that I've been working on, on and off for over a year, which would address this problem. I'm not sure when or if I'll finish it. It touches a lot of things, and my branch won't merge cleanly any more.

Additionally, the idea of moving f-strings into the Python grammar would also fix this problem. That might be a better way forward, although it has a number of downsides, too.

> Additionally, the idea of moving f-strings into the Python grammar would also fix this problem. That might be a better way forward, although it has a number of downsides, too.

For considerin this path we would need to wait to see first the future of the new experiment with the PEG parser to avoid potentially throwing away all the work in the parser and the tokenizer. Also, this approach will likely entail much more changes than reworking the error reporting code.

Yes, moving f-strings to the grammar would be a huge change, and not just for CPython.

I discussed it at the last PyCon with the authors of various editors (for syntax highlighting) and other tools that parse python code. No one was hugely opposed to it, and I think even one person was very excited about it. But it's not a step to be taken lightly. I don't think I've discussed it with any other Python implementors outside of CPython.

My biggest concern is that it makes naive string recognition fail. For example, this would become a valid f-string:
f'{fn('some string')}'
while
'{fn('some string')}'
is not a valid string.

I now I think I've completely derailed this bug report. I think the action item here is for me to finish up my better error reporting.

New changeset f7b1e461567e5e3fa3ba46f589d9edc1b45b2dd0 by Lysandros Nikolaou in branch 'master':
bpo-38964: Print correct filename on a SyntaxError in an fstring (GH-20399)
https://github.com/python/cpython/commit/f7b1e461567e5e3fa3ba46f589d9edc1b45b2dd0

New changeset 791a46ed58f74d673cf2c0d81deec57155bf7583 by Lysandros Nikolaou in branch '3.9':
[3.9] bpo-38964: Print correct filename on a SyntaxError in an fstring (GH-20399) (GH-20404)
https://github.com/python/cpython/commit/791a46ed58f74d673cf2c0d81deec57155bf7583