This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast unparse does not support f-string new debug format.
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: eric.smith, mbussonn, pablogsal
Priority: normal Keywords: patch

Created on 2019-05-21 22:03 by mbussonn, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13475 closed mbussonn, 2019-05-21 22:06
Messages (11)
msg343105 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2019-05-21 22:03
All in title, f"{x=}" implemented in https://bugs.python.org/issue36817
are not liked by unparse:  https://github.com/python/cpython/pull/13473,
msg343128 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-05-21 23:36
I don’t think this is worth changing. I’m planning on removing the expr_text field before beta 1. At which point I don’t think there will be any work to do here. 

See https://mail.python.org/pipermail/python-dev/2019-May/157493.html

Also, can you explain what’s currently broken? I’ll need to see some example code that doesn’t work in order to understand your change and if the ast change will affect it.
msg343131 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-05-21 23:48
Never mind on the failing code. I see your test cases now.
msg343144 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2019-05-22 04:34
Just for clarification for future readers;

I landed a PR that used f"{x=}" in some new functionalities, and that broke the buildbots, the buildbots do test that many of the `Lib/` files can be round-tripped ast->unparse->ast, and as ast-unparse did not understood f-debug, if roundtripped f"{x=}" to f"{x!r}". The PRs CIs did not fail as apparently they don't test this functionality.

So currently not having support for f-string-debug in ast-unparse kind of prevent using f-string debug format in the stdlib.

I'm happy if `expr_text` get removed, and having unparse round-trip to `x={x!r}` sounds sufficient to me; it may make like of formatters like black a bit harder maybe ? Though I believe black is likely not using ast but lib2to3.
msg343153 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-05-22 08:20
Thanks for the recap, Matthias. Once I remove expr_text (hopefully this weekend!), we should still at least add tests to make sure nothing fails, even with the ast-expanded version of the = f-strings.
msg343212 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-05-22 16:22
Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}' when unparsed as it compares the text of the file and the text of the roundtrip of the ast of the file
msg343222 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2019-05-22 17:47
I thought it was comparing the AST of the file to the AST of the unparsed
AST.

So it's actually checking if parse(unparse(x)) is indempotent and not
wether parse(unparse(x)) is indempotent.

So x={x!r} should be fine.

On Wed, May 22, 2019, 09:22 Pablo Galindo Salgado <report@bugs.python.org>
wrote:

>
> Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:
>
> Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}' when
> unparsed as it compares the text of the file and the text of the roundtrip
> of the ast of the file
>
> ----------
> nosy: +pablogsal
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue37003>
> _______________________________________
>
msg343226 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-05-22 18:13
Actually, it checks if the dump is the same:

 class ASTTestCase(unittest.TestCase):
    def assertASTEqual(self, ast1, ast2):
        self.assertEqual(ast.dump(ast1), ast.dump(ast2))

    def check_roundtrip(self, code1, filename="internal"):
        ast1 = compile(code1, filename, "exec", ast.PyCF_ONLY_AST)
        unparse_buffer = io.StringIO()
        unparse.Unparser(ast1, unparse_buffer)
        code2 = unparse_buffer.getvalue()
        ast2 = compile(code2, filename, "exec", ast.PyCF_ONLY_AST)
        self.assertASTEqual(ast1, ast2)
msg343227 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-05-22 18:17
> Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}'

I arrived at the wrong conclusion as Matthias points out.

>>> import ast
>>> ast.dump(compile("f'{x=}'","<string>","exec",ast.PyCF_ONLY_AST))
"Module(body=[Expr(value=JoinedStr(values=[FormattedValue(value=Name(id='x', ctx=Load()), conversion=114, format_spec=None, expr_text='x=')]))], type_ignores=[])"
>>> ast.dump(compile("f'{x!r}'","<string>","exec",ast.PyCF_ONLY_AST))
"Module(body=[Expr(value=JoinedStr(values=[FormattedValue(value=Name(id='x', ctx=Load()), conversion=114, format_spec=None, expr_text=None)]))], type_ignores=[])"

if expr_text is removed those strings will be the same, so we will be ok. Sorry for the confusion.
msg343725 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-05-28 01:23
mbussonn: can you re-check this in light of the changes for issue37050? I'm not sure if there's still a problem that needs fixing or not.
msg343726 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2019-05-28 01:39
I believe all is fine now. Thanks !
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81184
2019-05-28 01:39:45mbussonnsetstatus: open -> closed
resolution: fixed
messages: + msg343726

stage: patch review -> resolved
2019-05-28 01:23:28eric.smithsetmessages: + msg343725
2019-05-22 18:17:25pablogsalsetmessages: + msg343227
2019-05-22 18:13:52pablogsalsetmessages: + msg343226
2019-05-22 17:47:18mbussonnsetmessages: + msg343222
2019-05-22 16:22:31pablogsalsetnosy: + pablogsal
messages: + msg343212
2019-05-22 08:20:15eric.smithsetmessages: + msg343153
2019-05-22 04:34:07mbussonnsetmessages: + msg343144
2019-05-21 23:48:47eric.smithsetmessages: + msg343131
2019-05-21 23:36:35eric.smithsetmessages: + msg343128
2019-05-21 22:24:46larrysetassignee: eric.smith

nosy: + eric.smith
2019-05-21 22:06:38mbussonnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request13387
2019-05-21 22:03:08mbussonncreate