This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast: Different FormattedValue expressions have same col_offset information
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: BTaskaya, Weijar Z, eric.smith, lys.nikolaou
Priority: normal Keywords:

Created on 2019-07-01 02:10 by Weijar Z, last changed 2022-04-11 14:59 by admin.

Messages (5)
msg346950 - (view) Author: Weijar Z (Weijar Z) Date: 2019-07-01 02:10
This express 

  f"{x}{x}{y}"

will produce ast tree:

{
    "$node": "Module",
    "body": [
        {
            "$node": "Expr",
            "value": {
                "$node": "JoinedStr",
                "values": [
                    {
                        "$node": "FormattedValue",
                        "value": {
                            "$node": "Name",
                            "id": "x",
                            "ctx": {
                                "$node": "Load"
                            },
                            "lineno": 1,
                            "col_offset": 3
                        },
                        "conversion": -1,
                        "format_spec": null,
                        "lineno": 1,
                        "col_offset": 0
                    },
                    {
                        "$node": "FormattedValue",
                        "value": {
                            "$node": "Name",
                            "id": "x",
                            "ctx": {
                                "$node": "Load"
                            },
                            "lineno": 1,
                            "col_offset": 3
                        },
                        "conversion": -1,
                        "format_spec": null,
                        "lineno": 1,
                        "col_offset": 0
                    },
                    {
                        "$node": "FormattedValue",
                        "value": {
                            "$node": "Name",
                            "id": "y",
                            "ctx": {
                                "$node": "Load"
                            },
                            "lineno": 1,
                            "col_offset": 9
                        },
                        "conversion": -1,
                        "format_spec": null,
                        "lineno": 1,
                        "col_offset": 0
                    }
                ],
                "lineno": 1,
                "col_offset": 0
            },
            "lineno": 1,
            "col_offset": 0
        }
    ]
}

These two variable 'x' has same col_offset '3', is it wrong?
msg346981 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-07-01 08:58
I'm working on overhauling how these are calculated. But it's complex, and is taking a while.
msg372989 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-07-04 11:31
I still see this problem with 3.10, which I thought might have fixed this.

@lys.nikolaou: any ideas on this?
msg372991 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2020-07-04 13:24
> I still see this problem with 3.10, which I thought might have fixed this.
Nope, that's still true in 3.10.

>I'm working on overhauling how these are calculated. But it's complex, and is taking a while.
In short, the FormattedValue nodes all have the exact same lineno's and col_offset's, which are also identical to those of the enclosing JoinedStr node. These values are equal to lineno and col_offset of the first STRING token and end_lineno and end_col_offset of the last STRING token (note that the grammar accepts a STRING+ node).

> any ideas on this?
Moving f-string parsing to the parser, which we've been talking about lately, would solve this. But this will probably take some time, since I currently do not have the time. It's probably going to be a good project for this coming fall.

Another alternative, in case we don't want to wait until then, would be for the handwritten f-string parser to have its own instances of a lineno and col_offset, so that they can be used when the FormattedValue nodes are created. This would probably also require some effort though, so I'm not sure we want to do it, before we really know if we're gonnna proceed with the "moving f-string parsing to PEG" project.
msg373002 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-07-04 23:02
I think waiting until we decide what to do with the parser makes sense. This problem has been around for a while, and while it's unfortunate I don't think it's worth heroic measures to fix.
History
Date User Action Args
2022-04-11 14:59:17adminsetgithub: 81639
2020-07-04 23:02:28eric.smithsetmessages: + msg373002
2020-07-04 14:46:18BTaskayasetnosy: + BTaskaya
2020-07-04 13:24:27lys.nikolaousetmessages: + msg372991
2020-07-04 11:31:54eric.smithsetversions: + Python 3.10, - Python 3.7, Python 3.8
nosy: + lys.nikolaou

messages: + msg372989

components: + Interpreter Core, - Library (Lib)
2019-07-01 08:58:38eric.smithsetassignee: eric.smith

messages: + msg346981
nosy: + eric.smith
2019-07-01 02:10:41Weijar Zcreate