classification
Title: Code and IDLE should catch all compile errors.
Type: behavior Stage: test needed
Components: IDLE, Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: ppperry, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2015-11-25 14:26 by ppperry, last changed 2019-09-20 21:08 by terry.reedy.

Files
File name Uploaded Description Edit
catch_all_syntax_errors.diff terry.reedy, 2016-06-11 23:39 review
Messages (10)
msg255358 - (view) Author: (ppperry) Date: 2015-11-25 14:26
The following code:
for a in range(26):
  for b in range(26):
   for c in range(26):
    for d in range(26):
     for e in range(26):
      for f in range(26):
       for g in range(26):
        for h in range(26):
         for i in range(26):
          for j in range(26):
           for k in range(26):
            for l in range(26):
             for m in range(26):
              for o in range(26):
               for p in range(26):
                for q in range(26):
                 for r in range(26):
                  for s in range(26):
                   for t in range(26):
                    for u in range(26):
                     for v in range(26):
                       for w in range(26):
                         pass

fails to compile with `SystemError: too many statically nested blocks` when typed in the standard interpreter.

When typed in an IDLE shell , pressing enter will not try to run the code, instead printing the SystemError to the console from which idle was started, instead of to the IDLE shell, which would be expected.
msg255478 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-27 19:40
I believe startup files are compiled in PyShell.py, line 649 (3.x)
        try:
            code = compile(source, filename, "exec")
        except (OverflowError, SyntaxError):

and editor code is compiled in ScriptBinding.py, line 100
        try:
            # If successful, return the compiled code
            return compile(source, filename, "exec")
        except (SyntaxError, OverflowError, ValueError) as value:

Adding SystemError to the tuple is trivial.  I just need to test to verify.  (Side note: I think the tuple should be the same for both.  The reasons for entries other than SyntaxError should be documented.  ValueError when compiling?)

I believe that interactive input is forwarded to stdlib code.InteractiveInterpreter in PyShell.py, line 679
    try:
        # InteractiveInterpreter.runsource() calls its runcode() method,
        # which is overridden (see below)
        return InteractiveInterpreter.runsource(self, source, filename)

The compile call and error catching is inside Lib/code.py, line 57.
        try:
            code = self.compile(source, filename, symbol)
        except (OverflowError, SyntaxError, ValueError):

It should be fixed here.
msg255483 - (view) Author: (ppperry) Date: 2015-11-27 20:19
A similar problem occurs with the TypeError produced by attempting to compile a null byte.
msg255486 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-27 21:09
How lovely ;-)  

>>> compile(chr(0), '', 'eval')  # 2.7, chr(0) is byte
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    compile(chr(0), '', 'eval')
TypeError: compile() expected string without null bytes

>>> compile(chr(0), '', 'eval')  # 3.4, chr(0) is unichar
Traceback (most recent call last):
  File "<pyshell#25>", line 1, in <module>
    compile(chr(0), '', 'eval')
ValueError: source code string cannot contain null bytes
>>> compile(bytes(1), '', 'eval')  # == 2.7 chr(0)
Traceback (most recent call last):
  File "<pyshell#27>", line 1, in <module>
    compile(bytes(1), '', 'eval')
ValueError: source code string cannot contain null bytes

I almost think the long tuple should be replaced by Exception.
msg255487 - (view) Author: (ppperry) Date: 2015-11-27 21:14
Why does compile not support null bytes in the first place?
msg268171 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-06-11 01:53
How to cause
SyntaxError -- obvious
NameError -- ?, already caught in code module
OverflowError-- ?, already caught in code module
SystemError  - 22 nested for loops ('deeply nested blocks')
TypeError -- chr(0), 2.7
ValueError -- chr(0), 3.x; bytes(0), 2.7
msg268177 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-11 04:38
SystemError is serious bug. Please open separate issue for this.

> Why does compile not support null bytes in the first place?

Due to implementation detail of CPython tokenizer. I uses NUL-terminated C strings.

Yet one possible exception: MemoryError. I agree that the long tuple should be replaced by Exception.
msg268288 - (view) Author: (ppperry) Date: 2016-06-11 22:29
OverflowError is raised when one attempts to compile a string of 2**31 or more characters.
msg268295 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-06-11 23:39
Code patch, still needs tests.  ppperry, thanks.  I may create a temporary big file when large memory tests are allowed.
msg270406 - (view) Author: (ppperry) Date: 2016-07-14 12:35
>SystemError is serious bug. Please open separate issue for this.

Done. I created issue27514
History
Date User Action Args
2019-09-20 21:08:22terry.reedysetversions: + Python 3.9, - Python 2.7, Python 3.5, Python 3.6
2016-07-14 12:35:59ppperrysetmessages: + msg270406
2016-06-11 23:39:06terry.reedysetfiles: + catch_all_syntax_errors.diff
keywords: + patch
messages: + msg268295

stage: needs patch -> test needed
2016-06-11 22:29:13ppperrysetmessages: + msg268288
2016-06-11 04:38:12serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg268177
2016-06-11 01:53:07terry.reedysettitle: Idle does not handle non-standard compile errors -> Code and IDLE should catch all compile errors.
messages: + msg268171
components: + Library (Lib), - Interpreter Core
versions: - Python 3.4
2015-11-27 21:14:42ppperrysetmessages: + msg255487
2015-11-27 21:09:37terry.reedysetmessages: + msg255486
2015-11-27 20:19:51ppperrysetmessages: + msg255483
2015-11-27 19:40:42terry.reedysetassignee: terry.reedy
stage: needs patch
versions: + Python 3.4, Python 3.5, Python 3.6
2015-11-27 19:40:18terry.reedysetnosy: + terry.reedy
messages: + msg255478
2015-11-25 14:29:40ppperrysettype: behavior
2015-11-25 14:26:38ppperrycreate