This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Nested multi-line expression will lead to "compile()" fails
Type: compile error Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, hongweipeng, lukasz.langa, lys.nikolaou, miss-islington, pablogsal, xxm
Priority: normal Keywords: patch

Created on 2021-01-13 06:01 by xxm, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29934 merged hongweipeng, 2021-12-06 07:47
PR 30040 merged miss-islington, 2021-12-10 23:44
Messages (9)
msg384995 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 06:01
For build-in function compile() in mode 'single',  only single statement can be well compiled. If we compile multiple statements,  the parser will raise a syntaxError. Seeing the following two programs,  In program 1, 2 statement are compiled. So the parser raises a Syntax error. It's the expected output. However, if we insert a nested multi-line assignment in this code(see program 2), 3 statements are compiled. We expect the parser also raise a Sytax error. But it' not.

Program 1 ( with expected results)
===================================
code1 = """
a = 1
b = 2
"""
c = compile(code1, "", "single")
===================================
Traceback (most recent call last):
  File "/home/xxm/Desktop/nameChanging/report/test1.py", line 641, in <module>
    c = compile(code1, "", "single")
  File "", line 2
    a = 1
         ^
SyntaxError: multiple statements found while compiling a single statement


Program 2 (with unexpected results)
=================================
code2 = """
c ='''
d=1
'''
a = 1
b = 2
"""
c = compile(code2, "", "single")
=================================


Expected out for program 2: Raise a syntaxError too, But it's not.

>> python -V
Python 3.10.0a2
>>uname -a
Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
msg384999 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-13 06:35
What does the bytecode for c in program 2 look like?

Also, how do you find all these bugs? Are you developing a new fuzzier?
msg385000 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 07:04
The bytecode of c is as following:

              0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (d)
              4 LOAD_CONST               1 (None)
              6 RETURN_VALUE
===============================================

Yes, my team is working on developing a new fuzzier for the CPython.

Also, we are trying to apply it on the newest version of CPython. 

We will carefully analyze all bugs before reporting them on the tracker.

Hope our work does not bother you too much.
msg385001 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-13 07:11
It’s good work! Is it open source?
msg385003 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 07:30
For sure! As soon as we validate this technique, we will open source it on GitHub.
msg386542 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2021-02-05 23:11
Thanks a lot for the report, Xinmeng! This is a bug in how we're checking for bad single statements in bad_single_statement and it's not trivial to fix due to the string shenanigans we're doing there in newline_in_string, which does not account for many edge cases. I'll try to dig a bit deeper during the weekend.
msg408269 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-12-10 23:44
New changeset 28179aac796ed1debdce336c4b8ca18e8475d40d by Weipeng Hong in branch 'main':
bpo-42918: Improve build-in function compile() in mode 'single' (GH-29934)
https://github.com/python/cpython/commit/28179aac796ed1debdce336c4b8ca18e8475d40d
msg409231 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-12-27 16:15
New changeset 576e38f9db61ca09ca6dc57ad13b02a7bf9d370a by Miss Islington (bot) in branch '3.10':
bpo-42918: Improve built-in function compile() in mode 'single' (GH-29934) (GH-30040)
https://github.com/python/cpython/commit/576e38f9db61ca09ca6dc57ad13b02a7bf9d370a
msg409232 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-12-27 16:17
Thanks! ✨ 🍰 ✨
History
Date User Action Args
2022-04-11 14:59:40adminsetgithub: 87084
2021-12-27 16:17:23lukasz.langasetstatus: open -> closed
versions: + Python 3.11
messages: + msg409232

resolution: fixed
stage: patch review -> resolved
2021-12-27 16:15:59lukasz.langasetmessages: + msg409231
2021-12-10 23:44:38miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request28266
2021-12-10 23:44:34lukasz.langasetnosy: + lukasz.langa
messages: + msg408269
2021-12-06 07:47:19hongweipengsetkeywords: + patch
nosy: + hongweipeng

pull_requests: + pull_request28158
stage: patch review
2021-02-05 23:11:24lys.nikolaousetnosy: + pablogsal
messages: + msg386542
2021-01-13 07:30:24xxmsetmessages: + msg385003
2021-01-13 07:11:01gvanrossumsetmessages: + msg385001
2021-01-13 07:04:43xxmsetmessages: + msg385000
2021-01-13 06:35:45gvanrossumsetnosy: + gvanrossum
messages: + msg384999
2021-01-13 06:32:59gvanrossumsetnosy: + lys.nikolaou
2021-01-13 06:05:59xxmsetcomponents: + Interpreter Core
2021-01-13 06:01:57xxmcreate