Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Location of SyntaxError with new parser missing (after continuation character) #87721

Closed
aroberge mannequin opened this issue Mar 19, 2021 · 6 comments
Closed

Location of SyntaxError with new parser missing (after continuation character) #87721

aroberge mannequin opened this issue Mar 19, 2021 · 6 comments
Labels
3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@aroberge
Copy link
Mannequin

aroberge mannequin commented Mar 19, 2021

BPO 43555
Nosy @gvanrossum, @terryjreedy, @aroberge, @ammaraskar, @lysnikolaou, @pablogsal, @miss-islington
PRs
  • bpo-43555: Report the column offset for invalid line continuation character #24939
  • [3.9] bpo-43555: Report the column offset for invalid line continuation character (GH-24939) #24975
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-03-22.19:07:24.443>
    created_at = <Date 2021-03-19.11:18:52.239>
    labels = ['interpreter-core', 'type-bug', '3.9', '3.10']
    title = 'Location of SyntaxError with new parser missing (after continuation character)'
    updated_at = <Date 2021-03-22.23:53:31.254>
    user = 'https://github.com/aroberge'

    bugs.python.org fields:

    activity = <Date 2021-03-22.23:53:31.254>
    actor = 'ammar2'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-03-22.19:07:24.443>
    closer = 'pablogsal'
    components = ['Interpreter Core']
    creation = <Date 2021-03-19.11:18:52.239>
    creator = 'aroberge'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 43555
    keywords = ['patch']
    message_count = 6.0
    messages = ['389071', '389333', '389335', '389351', '389353', '389354']
    nosy_count = 7.0
    nosy_names = ['gvanrossum', 'terry.reedy', 'aroberge', 'ammar2', 'lys.nikolaou', 'pablogsal', 'miss-islington']
    pr_nums = ['24939', '24975']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue43555'
    versions = ['Python 3.9', 'Python 3.10']

    @aroberge
    Copy link
    Mannequin Author

    aroberge mannequin commented Mar 19, 2021

    Normally, for SyntaxErrors, the location of the error is indicated by a ^. There is at least one case where the location is missing for 3.9 and 3.10.0a6 where it was shown before. Using the old parser for 3.9, or with previous versions of Python, the location is shown.

    Python 3.10.0a6 ... on win32
    >>> a = 3 \ 4
      File "<stdin>", line 1
    SyntaxError: unexpected character after line continuation character
    >>>
    
    
    Python 3.9.0 ... on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> a = 3 \ 4
      File "<stdin>", line 1
    SyntaxError: unexpected character after line continuation character
    >>>

    Using the old parser with Python 3.9, the location of the error is shown *after* the unexpected character.

    > python -X oldparser
    Python 3.9.0 ... on win32
    >>> a = 3 \ 4
      File "<stdin>", line 1
        a = 3 \ 4
                 ^
    SyntaxError: unexpected character after line continuation character
    >>>

    Using Python 3.8 (and 3.7, 3.6), the location is pointing at the unexpected character.

    Python 3.8.4 ... on win32
    >>> a = 3 \ 4
      File "<stdin>", line 1
        a = 3 \ 4
                ^
    SyntaxError: unexpected character after line continuation character
    >>>

    @aroberge aroberge mannequin added 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Mar 19, 2021
    @pablogsal
    Copy link
    Member

    New changeset 96eeff5 by Pablo Galindo in branch 'master':
    bpo-43555: Report the column offset for invalid line continuation character (GH-24939)
    96eeff5

    @pablogsal
    Copy link
    Member

    New changeset 994a519 by Miss Islington (bot) in branch '3.9':
    bpo-43555: Report the column offset for invalid line continuation character (GH-24939) (bpo-24975)
    994a519

    @terryjreedy
    Copy link
    Member

    Before the patch, IDLE highlighted the \n endline with a red background, which tk displays as red background from the blank space after the 4 to the right edge of the text widget, including in 3.8.8.

    The 3.8 result, different from REPL, is due the the difference of using code._maybe_compile. The latter catches the SyntaxError for "3 \\ 4", which has offset 5, recompiles "3 \\ 4\n", and raises the subsequent SyntaxError, which has offset 6. (In this particular case where the message remains the same, the original SyntaxError instance with the original offset should have been kept and raised.)

    I have occasionally made this typing mistake and found the long red line slightly annoying, but never thought to compare it to the REPL caret or report it as a bug. The new parser made no difference in IDLE to newly annoy.

    After the patch, the (1-based) offset stays 5, which IDLE knows means the 5th char and hence it now highlights the '4'. Much better. Thank you both for the report and fix.

    Side question: https://docs.python.org/3/library/exceptions.html#SyntaxError says "Instances of this class have attributes filename, lineno, offset and text ...". Should it be documented that lineno and offset are both 1-based? Are these CPython accidents or part of the language? 1-based line numbers can be expected as common across languages, tk and python included. I believe that 1-based column offsets were viewed in a previous issue as a bug that we would not fix.

    @gvanrossum
    Copy link
    Member

    We should definitely document the column offset being 1-based, if it hasn't been already. But be careful, there are some APIs that are 0-based and others that are 1-based, for column offsets.

    I was quite surprised at some point to find out that we were inconsistent with the column offset, and then I looked at what Emacs and vim do, and I found that both interpret column offsets (in the familiar "filename:lineno:offset: message" format) as 1-based. IIRC I had to fix it in quite a few places because we were actually being inconsistent.

    @ammaraskar
    Copy link
    Member

    We should definitely document the column offset being 1-based

    Yes please, I remember working on that issue to make it consistently 1-based a while ago and I remember that the tooling was relying on 1-based indexes for column offsets.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants