classification
Title: syntax error in multiline f-string produces ~40k spaces output
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, asmeurer, eric.smith, pablogsal, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2019-06-27 18:22 by Anthony Sottile, last changed 2019-08-21 12:13 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 14433 merged Anthony Sottile, 2019-06-27 22:03
PR 15001 merged miss-islington, 2019-07-29 14:00
PR 15003 merged Anthony Sottile, 2019-07-29 14:24
PR 15005 merged miss-islington, 2019-07-29 15:12
Messages (12)
msg346767 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-06-27 18:22
For instance this file:

foo = f"""{}
foo"""

$ python3.8 --version --version
Python 3.8.0b1 (default, Jun  6 2019, 03:44:52) 
[GCC 7.4.0]

$ python3.8 t.py | wc -c
  File "t.py", line 1


<<snipped, but very very many spaces>>

                                   ^
SyntaxError: f-string: empty expression not allowed


$ python3.8 t.py |& wc -c
49134
msg346771 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-06-27 20:10
I can also trigger this with:

```
1+ """\q
"""
```

And `python -Werror`

so it seems not necessarily related to fstrings
msg346773 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-06-27 20:18
Debugging further, there's a token with a `multi_line_start` pointing at uninitialized memory -- let's see if I can't track down where that gets set and fix it :)
msg346777 - (view) Author: Aaron Meurer (asmeurer) Date: 2019-06-27 21:43
This looks like the same issue I mentioned here https://bugs.python.org/msg344764
msg346872 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-06-28 22:29
On Windows 10 with current 3.8 I see correct output, so the bug seems OS specific, even though the simple fix (see PR) is not.

C:\Users\Terry>py f:/python/a/tem4.py
  File "f:/python/a/tem4.py", line 1
    f'''{}
    ^
SyntaxError: f-string: empty expression not allowed
msg346873 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-06-28 22:31
just an f-string doesn't trigger this, there needs to be tokens before it
msg346874 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-06-28 23:51
OK, very obnoxious.  Fix restores behavior above.
msg348662 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2019-07-29 14:02
Thanks Anthony Sottile for the fix!

I suggest adding more test cases that do not involve f-strings, what do you think? Can you create another PR for those?
msg348664 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-07-29 14:06
sure! the only other case I could come up with involves multi-line strings and invalid escape sequences + Werror

let me see if I can fold that in as well
msg348938 - (view) Author: Aaron Meurer (asmeurer) Date: 2019-08-03 03:07
This seems related. It's also possible I'm misunderstanding what is supposed to happen here.

If you create test.py with just the 2 lines:

"""
a

and run python test.py from CPython master, you get

$./python.exe test.py
  File "/Users/aaronmeurer/Documents/cpython/test.py", line 4
    a
    ^
SyntaxError: EOF while scanning triple-quoted string literal

Notice that it reports line 4 even though the file only has 2 lines.

The offset in the syntax error is 6 columns (line numbers and column offsets
in SyntaxErrors count from 1)

>>> try:
...     compile('"""\na', '<none>', 'exec')
... except SyntaxError as e:
...     print(repr(e))
...
SyntaxError('EOF while scanning triple-quoted string literal', ('<none>', 2, 6, '"""\na\n'))
msg348962 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2019-08-03 14:24
seems unrelated and does the same on older versions of python:

```
$ python3.5 t.py
  File "t.py", line 4
    
    ^
SyntaxError: EOF while scanning triple-quoted string literal
```

I'd suggest a new issue or finding the dupe
msg350074 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-08-21 12:13
The initial issue has been fixed in master by the commit 5b94f3578c662d5f1ee90c0e6b81481d9ec82d89:

commit 5b94f3578c662d5f1ee90c0e6b81481d9ec82d89
Author: Anthony Sottile <asottile@umich.edu>
Date:   Mon Jul 29 06:59:13 2019 -0700

    Fix `SyntaxError` indicator printing too many spaces for multi-line strings (GH-14433)

Backport to 3.8: commit cf52bd0b9b1c1b7ecdd91e1ebebd15ae5c213a2c.

Thanks Anthony Sottile! I close the issue.

--

If there is a different issue in Python 3.7 and you consider that it must be fixed, please file a new issue. Python 3.5 and 3.6 don't accept bugfixes anymore:
https://devguide.python.org/#status-of-python-branches
History
Date User Action Args
2019-08-21 12:14:29vstinnerlinkissue37603 superseder
2019-08-21 12:13:50vstinnersetstatus: open -> closed

nosy: + vstinner
messages: + msg350074

resolution: fixed
stage: patch review -> resolved
2019-08-03 14:24:34Anthony Sottilesetmessages: + msg348962
2019-08-03 03:07:43asmeurersetmessages: + msg348938
2019-07-29 15:12:15miss-islingtonsetpull_requests: + pull_request14771
2019-07-29 14:24:06Anthony Sottilesetpull_requests: + pull_request14769
2019-07-29 14:06:12Anthony Sottilesetmessages: + msg348664
2019-07-29 14:02:52pablogsalsetnosy: + pablogsal
messages: + msg348662
2019-07-29 14:00:44miss-islingtonsetpull_requests: + pull_request14767
2019-06-28 23:51:32terry.reedysetmessages: + msg346874
2019-06-28 23:48:01terry.reedysetmessages: + msg346872
2019-06-28 23:47:36terry.reedysetmessages: - msg346872
2019-06-28 22:31:51Anthony Sottilesetmessages: + msg346873
2019-06-28 22:29:12terry.reedysetnosy: + terry.reedy

messages: + msg346872
title: syntax error in f-string in multiline string produces ~40k spaces of output -> syntax error in multiline f-string produces ~40k spaces output
2019-06-27 22:03:03Anthony Sottilesetkeywords: + patch
stage: patch review
pull_requests: + pull_request14250
2019-06-27 21:43:30asmeurersetnosy: + asmeurer
messages: + msg346777
2019-06-27 20:18:11Anthony Sottilesetmessages: + msg346773
2019-06-27 20:10:23Anthony Sottilesetmessages: + msg346771
2019-06-27 18:56:13serhiy.storchakasetnosy: + eric.smith
2019-06-27 18:22:41Anthony Sottilecreate