Title: Nested multi-line expression will lead to "compile()" fails
Type: compile error Stage:
Components: Interpreter Core Versions: Python 3.10
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, lys.nikolaou, pablogsal, xxm
Priority: normal Keywords:

Created on 2021-01-13 06:01 by xxm, last changed 2021-02-05 23:11 by lys.nikolaou.

Messages (6)
msg384995 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 06:01
For build-in function compile() in mode 'single',  only single statement can be well compiled. If we compile multiple statements,  the parser will raise a syntaxError. Seeing the following two programs,  In program 1, 2 statement are compiled. So the parser raises a Syntax error. It's the expected output. However, if we insert a nested multi-line assignment in this code(see program 2), 3 statements are compiled. We expect the parser also raise a Sytax error. But it' not.

Program 1 ( with expected results)
code1 = """
a = 1
b = 2
c = compile(code1, "", "single")
Traceback (most recent call last):
  File "/home/xxm/Desktop/nameChanging/report/", line 641, in <module>
    c = compile(code1, "", "single")
  File "", line 2
    a = 1
SyntaxError: multiple statements found while compiling a single statement

Program 2 (with unexpected results)
code2 = """
c ='''
a = 1
b = 2
c = compile(code2, "", "single")

Expected out for program 2: Raise a syntaxError too, But it's not.

>> python -V
Python 3.10.0a2
>>uname -a
Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
msg384999 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-13 06:35
What does the bytecode for c in program 2 look like?

Also, how do you find all these bugs? Are you developing a new fuzzier?
msg385000 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 07:04
The bytecode of c is as following:

              0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (d)
              4 LOAD_CONST               1 (None)
              6 RETURN_VALUE

Yes, my team is working on developing a new fuzzier for the CPython.

Also, we are trying to apply it on the newest version of CPython. 

We will carefully analyze all bugs before reporting them on the tracker.

Hope our work does not bother you too much.
msg385001 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-13 07:11
It’s good work! Is it open source?
msg385003 - (view) Author: Xinmeng Xia (xxm) Date: 2021-01-13 07:30
For sure! As soon as we validate this technique, we will open source it on GitHub.
msg386542 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2021-02-05 23:11
Thanks a lot for the report, Xinmeng! This is a bug in how we're checking for bad single statements in bad_single_statement and it's not trivial to fix due to the string shenanigans we're doing there in newline_in_string, which does not account for many edge cases. I'll try to dig a bit deeper during the weekend.
Date User Action Args
2021-02-05 23:11:24lys.nikolaousetnosy: + pablogsal
messages: + msg386542
2021-01-13 07:30:24xxmsetmessages: + msg385003
2021-01-13 07:11:01gvanrossumsetmessages: + msg385001
2021-01-13 07:04:43xxmsetmessages: + msg385000
2021-01-13 06:35:45gvanrossumsetnosy: + gvanrossum
messages: + msg384999
2021-01-13 06:32:59gvanrossumsetnosy: + lys.nikolaou
2021-01-13 06:05:59xxmsetcomponents: + Interpreter Core
2021-01-13 06:01:57xxmcreate