Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compile_command exec not raising syntax error with new PGEN Parser #84765

Closed
Carreau mannequin opened this issue May 10, 2020 · 3 comments
Closed

compile_command exec not raising syntax error with new PGEN Parser #84765

Carreau mannequin opened this issue May 10, 2020 · 3 comments
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@Carreau
Copy link
Mannequin

Carreau mannequin commented May 10, 2020

BPO 40585
Nosy @gvanrossum, @Carreau, @pablogsal
PRs
  • bpo-40585: Normalize errors messages in codeop when comparing them #20030
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-05-11.00:41:39.953>
created_at = <Date 2020-05-10.23:03:34.968>
labels = ['interpreter-core', '3.9']
title = 'compile_command exec not raising syntax error with new PGEN Parser'
updated_at = <Date 2020-05-11.03:40:31.514>
user = 'https://github.com/Carreau'

    bugs.python.org fields:

    activity = <Date 2020-05-11.03:40:31.514>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-11.00:41:39.953>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2020-05-10.23:03:34.968>
creator = 'mbussonn'
dependencies = []
files = []
hgrepos = []
issue_num = 40585
keywords = ['patch']
message_count = 3.0
messages = ['368594', '368597', '368600']
nosy_count = 3.0
nosy_names = ['gvanrossum', 'mbussonn', 'pablogsal']
pr_nums = ['20030']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40585'
versions = ['Python 3.9']
    @Carreau
    Copy link
    Mannequin Author

    Carreau mannequin commented May 10, 2020

    compile_command used to produce syntax error in exec mode:

    $ python -c "import codeop; codeop.compile_command('raise = 2', symbol='exec')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "...python3.8/codeop.py", line 125, in compile_command
    return _maybe_compile(_compile, source, filename, symbol)
  File "...python3.8/codeop.py", line 100, in _maybe_compile
    raise err1
  File "...python3.8/codeop.py", line 87, in _maybe_compile
    code1 = compiler(source + "\n", filename, symbol)
  File "...python3.8/codeop.py", line 105, in _compile
    return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
  File "<input>", line 1
    raise = 2
          ^
SyntaxError: invalid syntax

    This happens to not be the case anymore in master, where it simply return None for above invalid input.

    $ python -c "import codeop; codeop.CommandCompiler()('raise = 2', symbol='exec')"

    or many other:

     $ python
Python 3.9.0a6+ (heads/master:2cc9b8486d, May 10 2020, 15:52:00)
[Clang 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import codeop;
>>> codeop.compile_command('def a-b', symbol='exec')
>>> codeop.compile_command('await?', symbol='exec')
>>> codeop.compile_command('=!=', symbol='exec')
>>> codeop.compile_command('a await raise b', symbol='exec')
>>> codeop.compile_command('a await raise b?+1', symbol='exec')

    It is problematic as this is used in many places to decide whether code is valid, and for example in IPython to know wether we should insert a new line, or try to execute the code.

    It seem to be due to the new PGEN parser as setting PYTHONOLDPARSER solve the issue:

    $ PYTHONOLDPARSER=1 python -c "import codeop; codeop.CommandCompiler()('raise = 2', symbol='exec')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/bussonniermatthias/dev/cpython/Lib/codeop.py", line 171, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
  File "/Users/bussonniermatthias/dev/cpython/Lib/codeop.py", line 100, in _maybe_compile
    raise err1
  File "/Users/bussonniermatthias/dev/cpython/Lib/codeop.py", line 87, in _maybe_compile
    code1 = compiler(source + "\n", filename, symbol)
  File "/Users/bussonniermatthias/dev/cpython/Lib/codeop.py", line 136, in __call__
    codeob = compile(source, filename, symbol, self.flags, True)
  File "<input>", line 1
    raise = 2
          ^
SyntaxError: invalid syntax

    single and eval appear to behave fine.

    @Carreau Carreau mannequin added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels May 10, 2020
    @gvanrossum
    Copy link
    Member

    Let's analyze this here rather than in bpo-40334 (which is getting top-heavy :-).

    @pablogsal
    Copy link
    Member

    New changeset 5b956ca by Pablo Galindo in branch 'master':
    bpo-40585: Normalize errors messages in codeop when comparing them (GH-20030)
    5b956ca

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @gvanrossum @pablogsal