New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
codeop prematurely raises on 2nd line when closer is needed #87329
Comments
In Python3.10.0a5,
Similar things also happen in IDLE. |
IDLE is a Python program that wraps the Python interpreter. Before blaming IDLE for how code is executed, one should first execute the same code directly in Python. The example above shows that one should also try code.interact, especially for compilaition issues. (Xie, thank you for this reminder.) IDLE and code.interact use the same code and codeop infrastructure. Most relevant here is codeop.CommandCompiler, which should act the same as codeop.compile_command in the absence of __future__ imports. The codeop functions, including _maybe_compile, are intended to imitate in python the C REPL code that decides whether to execute, get more input, or raise. It doesn't always. (I think it would be good if a C expert compared it to the *current* C code.) >>> CC = codeop.CommandCompiler
>>> cc("print([1,") # Returns None, meaning, 'Get more input'.
>>> cc("print([1,\n2,")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "F:\dev\3x\lib\codeop.py", line 132, in compile_command
return _maybe_compile(_compile, source, filename, symbol)
File "F:\dev\3x\lib\codeop.py", line 106, in _maybe_compile
raise err1
File "F:\dev\3x\lib\codeop.py", line 93, in _maybe_compile
code1 = compiler(source + "\n", filename, symbol)
File "F:\dev\3x\lib\codeop.py", line 111, in _compile
return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
File "<input>", line 1
print([1,
^
SyntaxError: '[' was never closed If the opening '[' is removed, then ']' is replaced with (. If '[' is replaced with '{' and the items adjusted, ']' is replaced with '}'. In all three cases, CC should return None to ask for more, as the REPL does. |
test.test_codeop's test_incomplete is incomplete in that it tests several missing-closer single lines, like the above and "[x for x in (", all of which pass, but no corresponding multiple line snippets, like the above. The only multiple line snippets are for compound statements. As far as I could see, the same is true for test_valid. Are there similar tests for the REPL? Are they more complete? Should not both get the same tests? |
That would be very beneficial but is quite a big task, specially given that codeop design is quite surprising. I personally find quite confusing the way codeop is handling the decisions on continuing or not (is based on compiling the code with different levels of new lines and then checking the repr() of the exceptions). It would be good if someone familiar with "codeop" design can take a look on what's the best way to proceed. |
This issue is for 3.10 only since the never-closed message was only added for 3.10.0a5 and not backported. Pablo, Is there any sane possibility that the compiler could, at least sometimes, raise an IncompleteSyntax subclass of SyntaxError when code is merely incomplete, as opposed to wrong? Do you happen to know where the REPL code is? (If not, I will ask elsewhere.) |
I'm assuming you mean the parser (when the compiler runs the code must have been correct or otherwise the parser would have raised before). In case of the parser, I'm not sure but the answer is generally no. The parser doesn't have a concept of "partial Syntax" so that should be defined. There is also the problem that some of these syntax error come from the tokenizer which has its own rules. We could probably do more in the tokenizer than in the parser but this would add considerable complexity in the general case. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: