Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syntax error in multiline f-string produces ~40k spaces output #81614

Closed
asottile mannequin opened this issue Jun 27, 2019 · 12 comments
Closed

syntax error in multiline f-string produces ~40k spaces output #81614

asottile mannequin opened this issue Jun 27, 2019 · 12 comments
Labels
3.8 only security fixes 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@asottile
Copy link
Mannequin

asottile mannequin commented Jun 27, 2019

BPO 37433
Nosy @terryjreedy, @vstinner, @ericvsmith, @asottile, @pablogsal, @asmeurer
PRs
  • [bpo-37433] Fix SyntaxError indicator printing too many spaces for multi-line strings #14433
  • [3.8] bpo-37433: Fix SyntaxError indicator printing too many spaces for multi-line strings (GH-14433) #15001
  • bpo-37433: Add additional test for multi-line SyntaxError #15003
  • [3.8] bpo-37433: Add additional test for multi-line SyntaxError (GH-15003) #15005
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-08-21.12:13:50.094>
    created_at = <Date 2019-06-27.18:22:41.826>
    labels = ['interpreter-core', '3.8', '3.9']
    title = 'syntax error in multiline f-string produces ~40k spaces output'
    updated_at = <Date 2019-08-21.12:13:50.088>
    user = 'https://github.com/asottile'

    bugs.python.org fields:

    activity = <Date 2019-08-21.12:13:50.088>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-08-21.12:13:50.094>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2019-06-27.18:22:41.826>
    creator = 'Anthony Sottile'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 37433
    keywords = ['patch']
    message_count = 12.0
    messages = ['346767', '346771', '346773', '346777', '346872', '346873', '346874', '348662', '348664', '348938', '348962', '350074']
    nosy_count = 6.0
    nosy_names = ['terry.reedy', 'vstinner', 'eric.smith', 'Anthony Sottile', 'pablogsal', 'asmeurer']
    pr_nums = ['14433', '15001', '15003', '15005']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue37433'
    versions = ['Python 3.8', 'Python 3.9']

    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Jun 27, 2019

    For instance this file:

    foo = f"""{}
    foo"""
    $ python3.8 --version --version
    Python 3.8.0b1 (default, Jun  6 2019, 03:44:52) 
    [GCC 7.4.0]
    
    $ python3.8 t.py | wc -c
      File "t.py", line 1

    <<snipped, but very very many spaces>>

                                   ^
    

    SyntaxError: f-string: empty expression not allowed

    $ python3.8 t.py |& wc -c
    49134

    @asottile asottile mannequin added 3.8 only security fixes 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jun 27, 2019
    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Jun 27, 2019

    I can also trigger this with:

    1+ """\q
    """
    

    And python -Werror

    so it seems not necessarily related to fstrings

    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Jun 27, 2019

    Debugging further, there's a token with a multi_line_start pointing at uninitialized memory -- let's see if I can't track down where that gets set and fix it :)

    @asmeurer
    Copy link
    Mannequin

    asmeurer mannequin commented Jun 27, 2019

    This looks like the same issue I mentioned here https://bugs.python.org/msg344764

    @terryjreedy
    Copy link
    Member

    On Windows 10 with current 3.8 I see correct output, so the bug seems OS specific, even though the simple fix (see PR) is not.

    C:\Users\Terry>py f:/python/a/tem4.py
    File "f:/python/a/tem4.py", line 1
    f'''{}
    ^
    SyntaxError: f-string: empty expression not allowed

    @terryjreedy terryjreedy changed the title syntax error in f-string in multiline string produces ~40k spaces of output syntax error in multiline f-string produces ~40k spaces output Jun 28, 2019
    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Jun 28, 2019

    just an f-string doesn't trigger this, there needs to be tokens before it

    @terryjreedy
    Copy link
    Member

    OK, very obnoxious. Fix restores behavior above.

    @pablogsal
    Copy link
    Member

    Thanks Anthony Sottile for the fix!

    I suggest adding more test cases that do not involve f-strings, what do you think? Can you create another PR for those?

    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Jul 29, 2019

    sure! the only other case I could come up with involves multi-line strings and invalid escape sequences + Werror

    let me see if I can fold that in as well

    @asmeurer
    Copy link
    Mannequin

    asmeurer mannequin commented Aug 3, 2019

    This seems related. It's also possible I'm misunderstanding what is supposed to happen here.

    If you create test.py with just the 2 lines:

    """
    a

    and run python test.py from CPython master, you get

    $./python.exe test.py
    File "/Users/aaronmeurer/Documents/cpython/test.py", line 4
    a
    ^
    SyntaxError: EOF while scanning triple-quoted string literal

    Notice that it reports line 4 even though the file only has 2 lines.

    The offset in the syntax error is 6 columns (line numbers and column offsets
    in SyntaxErrors count from 1)

    >>> try:
    ...     compile('"""\na', '<none>', 'exec')
    ... except SyntaxError as e:
    ...     print(repr(e))
    ...
    SyntaxError('EOF while scanning triple-quoted string literal', ('<none>', 2, 6, '"""\na\n'))

    @asottile
    Copy link
    Mannequin Author

    asottile mannequin commented Aug 3, 2019

    seems unrelated and does the same on older versions of python:

    $ python3.5 t.py
      File "t.py", line 4
        
        ^
    SyntaxError: EOF while scanning triple-quoted string literal
    

    I'd suggest a new issue or finding the dupe

    @vstinner
    Copy link
    Member

    vstinner commented Aug 21, 2019

    The initial issue has been fixed in master by the commit 5b94f35:

    commit 5b94f3578c662d5f1ee90c0e6b81481d9ec82d89
    Author: Anthony Sottile \<asottile@umich.edu\>
    Date:   Mon Jul 29 06:59:13 2019 -0700
    
        Fix `SyntaxError` indicator printing too many spaces for multi-line strings (GH-14433)
    
    Backport to 3.8: commit cf52bd0b9b1c1b7ecdd91e1ebebd15ae5c213a2c.
    
    Thanks Anthony Sottile! I close the issue.
    

    If there is a different issue in Python 3.7 and you consider that it must be fixed, please file a new issue. Python 3.5 and 3.6 don't accept bugfixes anymore:
    https://devguide.python.org/#status-of-python-branches

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants