classification
Title: Incorrect exception highlighting for fstring format
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, ammar2, gvanrossum, lys.nikolaou, pablogsal
Priority: normal Keywords: patch

Created on 2021-08-11 07:59 by pablogsal, last changed 2021-08-12 17:46 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27729 merged pablogsal, 2021-08-11 15:39
PR 27743 merged pablogsal, 2021-08-12 16:21
PR 27744 merged pablogsal, 2021-08-12 16:25
Messages (8)
msg399372 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-11 07:59
Given this code:

print(f"Here is that pesky {xxx/2:.3f} again")

The traceback prints:

Traceback (most recent call last):
  File "/home/pablogsal/github/python/main/lel.py", line 1, in <module>
    print(f"Here is that pesky {xxx/2:.3f} again")
           ^^^
NameError: name 'xxx' is not defined

Removing the formatting part ":.3f" makes it work as expected
msg399392 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-08-11 13:30
(I originally reported this.)
msg399397 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-11 15:01
The problem lies here:

https://github.com/python/cpython/blob/f66d00fdd7e9a333accc6bf0e37173051aaa55d0/Parser/string_parser.c#L374-L389

The problem is that the strstr call will fail because the string containing the expression doesn't have the formatting part in it, so the offsets never get corrected.
msg399399 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-11 15:13
Actually, this has even more problems. Because we are using strstr to find the start of the expression in the parent string, if the expression is repeated the offsets are incorrectly generated:

For example:

print(f"Here is that {xxx} pesky {xxx} again")

This produces:

...
                     FormattedValue(
                        value=Name(
                           id='xxx',
                           ctx=Load(),
                           lineno=1,
                           col_offset=22,
                           end_lineno=1,
                           end_col_offset=25),
...
                     FormattedValue(
                        value=Name(
                           id='xxxx',
                           ctx=Load(),
                           lineno=1,
                           col_offset=22,
                           end_lineno=1,
                           end_col_offset=25),
...

while

print(f"Here is that {xxx} pesky {xxxx} again")

(different variables) produces:

...
                     FormattedValue(
                        value=Name(
                           id='xxx',
                           ctx=Load(),
                           lineno=1,
                           col_offset=22,
                           end_lineno=1,
                           end_col_offset=25),
...
                     FormattedValue(
                        value=Name(
                           id='xxxx',
                           ctx=Load(),
                           lineno=1,
                           col_offset=34,
                           end_lineno=1,
                           end_col_offset=38),
...
msg399400 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-11 15:24
Ha, we have test that knows is broken:

https://github.com/python/cpython/blob/f66d00fdd7e9a333accc6bf0e37173051aaa55d0/Lib/test/test_fstring.py#L217
msg399469 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-12 16:13
New changeset 8e832fb2a2cb54d7262148b6ec15563dffb48d63 by Pablo Galindo Salgado in branch 'main':
bpo-44885: Correct the ast locations of f-strings with format specs and repeated expressions (GH-27729)
https://github.com/python/cpython/commit/8e832fb2a2cb54d7262148b6ec15563dffb48d63
msg399471 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-12 16:41
New changeset c28c2e1cb0dd5fc8b17514e2c4ee17415af733a4 by Pablo Galindo Salgado in branch '3.10':
[3.10] bpo-44885: Correct the ast locations of f-strings with format specs and repeated expressions (GH-27729) (GH-27743)
https://github.com/python/cpython/commit/c28c2e1cb0dd5fc8b17514e2c4ee17415af733a4
msg399475 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-12 17:46
New changeset 4b86c9c5146c339c689830619be9d29b8f7bf417 by Pablo Galindo Salgado in branch '3.9':
[3.9] bpo-44885: Correct the ast locations of f-strings with format specs and repeated expressions (GH-27729) (GH-27744)
https://github.com/python/cpython/commit/4b86c9c5146c339c689830619be9d29b8f7bf417
History
Date User Action Args
2021-08-12 17:46:49pablogsalsetmessages: + msg399475
2021-08-12 16:42:19pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-08-12 16:41:25pablogsalsetmessages: + msg399471
2021-08-12 16:25:58pablogsalsetpull_requests: + pull_request26222
2021-08-12 16:21:08pablogsalsetpull_requests: + pull_request26221
2021-08-12 16:13:40pablogsalsetmessages: + msg399469
2021-08-11 15:39:17pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26209
2021-08-11 15:24:44pablogsalsetmessages: + msg399400
2021-08-11 15:13:10pablogsalsetmessages: + msg399399
2021-08-11 15:01:15pablogsalsetnosy: + lys.nikolaou
messages: + msg399397
2021-08-11 13:30:41gvanrossumsetnosy: + gvanrossum
messages: + msg399392
2021-08-11 07:59:02pablogsalcreate