Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pegen parser: Multiline eval with assignment to function call: assertion failed #86993

Closed
JulienPalard opened this issue Jan 4, 2021 · 8 comments
Closed

pegen parser: Multiline eval with assignment to function call: assertion failed #86993

JulienPalard opened this issue Jan 4, 2021 · 8 comments
Labels
3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@JulienPalard
Copy link
Member

BPO 42827
Nosy @gvanrossum, @terryjreedy, @vstinner, @JulienPalard, @lysnikolaou, @pablogsal
PRs
  • bpo-42827: Fix crash when determining the error location of multi-line expressions #24107
  • bpo-42827: Fix crash when storing error line of multi-line expressions #24124
  • bpo-42827: Fix crash on SyntaxError in multiline expressions #24140
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-01-14.22:34:41.123>
created_at = <Date 2021-01-04.20:48:09.775>
labels = ['interpreter-core', '3.10', '3.9', 'type-crash']
title = 'pegen parser: Multiline eval with assignment to function call:  assertion failed'
updated_at = <Date 2021-01-15.01:52:49.209>
user = 'https://github.com/JulienPalard'

    bugs.python.org fields:

    activity = <Date 2021-01-15.01:52:49.209>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-14.22:34:41.123>
closer = 'lys.nikolaou'
components = ['Interpreter Core']
creation = <Date 2021-01-04.20:48:09.775>
creator = 'mdk'
dependencies = []
files = []
hgrepos = []
issue_num = 42827
keywords = ['patch', '3.9regression']
message_count = 8.0
messages = ['384348', '384349', '384350', '384360', '385085', '385087', '385092', '385093']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'terry.reedy', 'vstinner', 'mdk', 'lys.nikolaou', 'pablogsal']
pr_nums = ['24107', '24124', '24140']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue42827'
versions = ['Python 3.9', 'Python 3.10']
    @JulienPalard
    Copy link
    Member Author

    Tested in 3.9.0 and master (127dde5):

    $ cat test.py
eval(
    """[file for str(file) in []
]
"""
)

$ ./python test.py
python: Parser/pegen.c:149: byte_offset_to_character_offset: Assertion `col_offset >= 0 && (unsigned long)col_offset <= strlen(str)' failed.
Fatal Python error: Aborted

    Current thread 0x00007f6454cfe280 (most recent call first):
    File "/home/mdk/clones/python/cpython/test.py", line 1 in <module>
    Aborted (core dumped)

    (Adding Guido and Pablo to nosy as, IIRC, you're interested in parser things :))

    @JulienPalard JulienPalard added 3.10 only security fixes type-crash A hard crash of the interpreter, possibly with a core dump labels Jan 4, 2021
    @vstinner
    Copy link
    Member

    vstinner commented Jan 4, 2021

    Reproducer as a single line:

    compile("[file for str(file) in []\n]", "string", "exec")

    @vstinner vstinner added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.9 only security fixes labels Jan 4, 2021
    @vstinner vstinner changed the title Multiline eval with assignment to function call segfaults pegen parser: Multiline eval with assignment to function call: assertion failed Jan 4, 2021
    @vstinner vstinner changed the title Multiline eval with assignment to function call segfaults pegen parser: Multiline eval with assignment to function call: assertion failed Jan 4, 2021
    @JulienPalard
    Copy link
    Member Author

    According to git bisect it come from 01ece63, thanks Victor for adding Lysandros, you had a good feeling :)

    @lysnikolaou
    Copy link
    Contributor

    Thanks for letting us know, Julien! I've identified the bug and where it comes from and I'll probably work on it tomorrow.

    @pablogsal
    Copy link
    Member

    New changeset e5fe509 by Lysandros Nikolaou in branch 'master':
    bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)
    e5fe509

    @lysnikolaou
    Copy link
    Contributor

    This is now fixed.

    @terryjreedy
    Copy link
    Member

    On Windows, Victor's one liner failed on repository debug builds with Windows' process error popup. I have seen this mysterious failure at least once when testing IDLE Shell running on the repository build. After the patch, my debug build gives the correct message,
    SyntaxError: cannot assign to function call
    and correctly points to the 's' of 'str' on the first line. Ditto when run from IDLE's Shell or editor. Thank you for the fix.

    Installed Python 3.10.0a4, Jan 4, gives the correct message but confusingly points at the final ']' on a line by itself. The latter should be fixed in the next release.

    @terryjreedy
    Copy link
    Member 

    In order to test code.InteractiveCompiler (used by IDLE) and codeop._maybe_compile (which IC calls), which calls compile() 3 times with mode 'single', I entered the example as 2 or 3 lines in Shell.
>>> [file for str(file) in []  # Error highlight on 's'.
 ]
SyntaxError: cannot assign to function call

    Breaking the expression elsewhere, including before 'str', gives the same message and highlighted 's'.

    I did not test before recompiling but it is certainly correct now.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    5 participants
    @vstinner @JulienPalard @pablogsal @terryjreedy @lysnikolaou