➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Tools/parser/unparse.py needs to be updated for f-strings
Type: behavior Stage: patch review
Components: Demos and Tools Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: eric.smith, martin.panter, python-dev
Priority: normal Keywords: easy, patch

Created on 2015-09-19 19:17 by eric.smith, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fstring-unparse.patch martin.panter, 2015-09-20 04:39 review
Messages (7)
msg251106 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2015-09-19 19:17
test_unparse.py occasionally fails if it picks a module that uses f-strings.
msg251107 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2015-09-19 19:21
And it always fails with "-u cpu", which the buildbots use.
msg251111 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-09-19 19:50
New changeset 37d3b95a289b by Eric V. Smith in branch 'default':
Temporary hack for issue #25180: exclude test_fstring.py from the unparse round-tripping, while I figure out how to properly fix it.
https://hg.python.org/cpython/rev/37d3b95a289b
msg251120 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2015-09-19 22:04
This task is actually pretty difficult, and is going to require some major surgery to unparse.py.

Unfortunately, until it's fixed, you can't use f-strings in the stdlib or in stdlib tests.

Particularly challenging are nested f-strings like:
f'{f"{0}"*3}'

Getting the quoting right will be hard, and will likely require passing another parameter around to all of the dispatch functions. Or maybe just some local state, similar to the indent level (_indent) which is already being tracked.
msg251137 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-09-20 04:39
I think this patch should do it. No major surgery required, just a good dose of recursion :)

def test_nested_fstrings(self):  # Original code
    y = 5
    self.assertEqual(f'{f"{0}"*3}', '000')
    self.assertEqual(f'{f"{y}"*3}', '555')
    self.assertEqual(f'{f"{\'x\'}"*3}', 'xxx')

    self.assertEqual(f"{r'x' f'{\"s\"}'}", 'xs')
    self.assertEqual(f"{r'x'rf'{\"s\"}'}", 'xs')

def test_nested_fstrings(self):  # Unparsed output
    y = 5
    self.assertEqual(f"{(f'{0}' * 3)}", '000')
    self.assertEqual(f"{(f'{y}' * 3)}", '555')
    self.assertEqual(f'{(f"{\'x\'}" * 3)}', 'xxx')
    self.assertEqual(f'{f"x{\'s\'}"}', 'xs')
    self.assertEqual(f'{f"x{\'s\'}"}', 'xs')

There was no problem getting the quoting right; repr() takes care of that, and then you slap the “f” on the front. The most subtle thing was knowing that f"{ {...} }" cannot be unparsed as f"{{...}}". I unparse other expressions without adding spaces, but unparse that one as f"{ {...}}".

Tested by running ./python -bWall -m test -u cpu test_tools
msg251161 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2015-09-20 09:58
That's awesome, thanks! Definitely simpler than where I was going.

I'm not in front of my dev machine right now, so I can't run it. But if it works, it works.

I suggest adding the test cases to test_unparse.py's UnparseTestCase. That way, any problems with this are caught earlier. The comment there says "Tests for specific bugs found in earlier versions of unparse", which this qualifies for!
msg251176 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-09-20 19:09
New changeset 2fcd2540ab97 by Eric V. Smith in branch 'default':
Issue 25180: Fix Tools/parser/unparse.py for f-strings. Patch by Martin Panter.
https://hg.python.org/cpython/rev/2fcd2540ab97
History
Date User Action Args
2022-04-11 14:58:21adminsetgithub: 69367
2015-09-20 19:10:18eric.smithsetstatus: open -> closed
resolution: fixed
2015-09-20 19:09:34python-devsetnosy: + python-dev
messages: + msg251176
2015-09-20 09:58:31eric.smithsetmessages: + msg251161
2015-09-20 04:39:45martin.pantersetfiles: + fstring-unparse.patch

nosy: + martin.panter
messages: + msg251137

keywords: + patch
stage: needs patch -> patch review
2015-09-19 22:04:09eric.smithsetnosy: - python-dev
messages: + msg251120
components: + Demos and Tools
2015-09-19 19:50:12python-devsetnosy: + python-dev
messages: + msg251111
2015-09-19 19:21:31eric.smithsetmessages: + msg251107
2015-09-19 19:17:23eric.smithcreate