Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error message differs when an expression is in an fstring #84448

Closed
lysnikolaou opened this issue Apr 12, 2020 · 3 comments
Closed

Error message differs when an expression is in an fstring #84448

lysnikolaou opened this issue Apr 12, 2020 · 3 comments
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@lysnikolaou
Copy link
Contributor

BPO 40267
Nosy @gvanrossum, @ericvsmith, @lysnikolaou, @pablogsal, @isidentical
PRs
  • bpo-40267: Fix message when last input character produces a SyntaxError #19521
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-04-15.18:24:17.588>
    created_at = <Date 2020-04-12.21:59:24.577>
    labels = ['interpreter-core', 'type-bug', '3.9']
    title = 'Error message differs when an expression is in an fstring'
    updated_at = <Date 2020-04-15.18:24:17.587>
    user = 'https://github.com/lysnikolaou'

    bugs.python.org fields:

    activity = <Date 2020-04-15.18:24:17.587>
    actor = 'gvanrossum'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-04-15.18:24:17.588>
    closer = 'gvanrossum'
    components = ['Interpreter Core']
    creation = <Date 2020-04-12.21:59:24.577>
    creator = 'lys.nikolaou'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40267
    keywords = ['patch']
    message_count = 3.0
    messages = ['366272', '366273', '366535']
    nosy_count = 5.0
    nosy_names = ['gvanrossum', 'eric.smith', 'lys.nikolaou', 'pablogsal', 'BTaskaya']
    pr_nums = ['19521']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40267'
    versions = ['Python 3.9']

    @lysnikolaou
    Copy link
    Contributor Author

    There are cases, where the error message differs, when an expression is being parsed inside an fstring. For example:

    >>> f'{x+}'
      File "<fstring>", line 1
        (x+)
           ^
    SyntaxError: unexpected EOF while parsing
    >>> (x+)
      File "<stdin>", line 1
        (x+)
           ^
    SyntaxError: invalid syntax

    Or with lambda definitions:

    >>> f'{lambda x:x}'
      File "<fstring>", line 1
        (lambda x)
                 ^
    SyntaxError: unexpected EOF while parsing
    >>> (lambda x)
      File "<stdin>", line 1
        (lambda x)
                 ^
    SyntaxError: invalid syntax

    @lysnikolaou lysnikolaou added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Apr 12, 2020
    @lysnikolaou
    Copy link
    Contributor Author

    It seems that this is actually a bit bigger than this and it is not specific to f-strings.

    The error message always changes to unexpected EOF while parsing if there is an error with the last character of the input and no newline follows. For example, as made clear to me by Guido, there are even differences in error messages between exec'ing and eval'ing something:

    >>> exec('x+')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1
        x+
         ^
    SyntaxError: invalid syntax
    >>> eval('x+')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1
        x+
         ^
    SyntaxError: unexpected EOF while parsing

    That's because the tokenizer adds an implicit newline to the input string, before tokenizing it, when the input comes from an exec call. (see

    /* If this is exec input, add a newline to the end of the string if
    )

    And that's not limited to a character missing, as suggested by the error message. Even when the last character itself generates a SyntaxError, the error message remains "unexpcted EOF while parsing":

    >>> x+@
      File "<stdin>", line 1
        x+@
          ^
    SyntaxError: invalid syntax
    >>> eval('x+@')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1
        x+@
          ^
    SyntaxError: unexpected EOF while parsing

    Thus, a very simple fix to the specific fstring problem of this issue would be to add a newline to the string that gets parsed to the parser in fstring_compile_expr in ast.c, but I guess it'd be better to fix the tokenizer itself, if this is considered a bug.

    @lysnikolaou lysnikolaou added the type-bug An unexpected behavior, bug, or error label Apr 13, 2020
    @gvanrossum
    Copy link
    Member

    New changeset 9a4b38f by Lysandros Nikolaou in branch 'master':
    bpo-40267: Fix message when last input character produces a SyntaxError (GH-19521)
    9a4b38f

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants