classification
Title: f-string assert is too restrictive
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: eric.smith, ned.deily, serhiy.storchaka, terry.reedy
Priority: normal Keywords:

Created on 2017-06-16 09:13 by eric.smith, last changed 2017-06-16 17:13 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2232 eric.smith, 2017-06-16 09:33
PR 2242 merged serhiy.storchaka, 2017-06-16 10:22
Messages (5)
msg296166 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-16 09:13
>>> eval("f'\\\n'")
python: Python/ast.c:4906: FstringParser_ConcatFstring: Assertion `!state->last_str || PyUnicode_GET_LENGTH(state->last_str) != 0' failed.
[2]    12810 abort (core dumped)  ./python

The problem is that some literal strings are zero length. The assert can be deleted.
msg296174 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-16 10:19
New changeset 11e97f2f80bf65cc828c127eafc95229df35d403 by Serhiy Storchaka (ericvsmith) in branch 'master':
bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (#2232)
https://github.com/python/cpython/commit/11e97f2f80bf65cc828c127eafc95229df35d403
msg296193 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-16 13:29
New changeset 2eca5b465f7404cc8484457b7966f828f434ec20 by ericvsmith (Serhiy Storchaka) in branch '3.6':
[3.6] bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (GH-2232) (#2242)
https://github.com/python/cpython/commit/2eca5b465f7404cc8484457b7966f828f434ec20
msg296204 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-16 15:35
Before the patch, eval("f'\\\n'") gives me the same assertion failure, with an IDLE freeze instead of core dump equivalent, but is seems that the string should be of length 2.

>>> len('\\\n')
2
>>> len(f'\\\n')
2
msg296212 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-06-16 16:49
Terry:

The eval is important. The bug was in evaluating an f-string that consisted of two bytes: a backslash followed by a newline.

And just as:
eval("'\\\n'") == ''  # len == 0
so should
eval("f'\\\n'") == ''  # len == 0

It's the second one that was throwing the assertion. The parser was seeing these bytes:
f  0x66
'  0x27
\  0x5c
nl 0xa
'  0x27
and behaving badly (to say the least) by asserting.

Without the eval, I can't think of a way to have a string consisting of those two bytes, but I assume someone who's trickier than I can come up with a way.
History
Date User Action Args
2017-06-16 17:13:05serhiy.storchakasettype: behavior -> crash
2017-06-16 16:49:50eric.smithsetstatus: open -> closed
resolution: fixed
messages: + msg296212
2017-06-16 15:35:28terry.reedysetnosy: + terry.reedy
messages: + msg296204

type: behavior
stage: resolved
2017-06-16 13:29:44eric.smithsetmessages: + msg296193
2017-06-16 10:22:59serhiy.storchakasetpull_requests: + pull_request2286
2017-06-16 10:19:34serhiy.storchakasetmessages: + msg296174
2017-06-16 09:33:11eric.smithsetpull_requests: + pull_request2285
2017-06-16 09:13:47eric.smithcreate