classification
Title: IDLE: Module warnings misplaced.
Type: behavior Stage: test needed
Components: IDLE Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: rhettinger, serhiy.storchaka, taleinat, terry.reedy
Priority: normal Keywords:

Created on 2018-09-30 20:10 by terry.reedy, last changed 2019-08-16 05:23 by terry.reedy.

Messages (7)
msg326745 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-09-30 20:10
In 3.6.6, """compile("assert (0, 'bad')", '', 'single')""" in Python or IDLE emits "SyntaxWarning: assertion is always true, perhaps remove parentheses?".

In Python,
>>> assert (0, 'bad')
<stdin>:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?

In IDLE's Shell, the same statement is effectively treated as incomplete in that newlines are echoed and IDLE waits for input until something is entered.  KeyboardInterrupt (^C) or any text terminate the loop.  In the latter case, "SyntaxError: multiple statements found while compiling a single statement" is printed.

In a file, the statement has no effect, but the warning should be printed in the shell.
msg334316 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-01-24 19:20
In 3.7.2, the statement in a file results in the SyntaxError being printed in Shell *before* the restart.  It should be after, but before the code object is sent to the restarted execution process.
msg349394 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-08-11 16:21
I cannot reproduce this behavior in 3.7+. `assert (0, 'bad')` does not have any effect. But in 3.6 it outputs a traceback to the stderr.

$ ./python -m idlelib
Exception in Tkinter callback
Traceback (most recent call last):
  File "/home/serhiy/py/cpython3.6/Lib/tkinter/__init__.py", line 1705, in __call__
    return self.func(*args)
  File "/home/serhiy/py/cpython3.6/Lib/idlelib/multicall.py", line 176, in handler
    r = l[i](event)
  File "/home/serhiy/py/cpython3.6/Lib/idlelib/pyshell.py", line 1205, in enter_callback
    self.runit()
  File "/home/serhiy/py/cpython3.6/Lib/idlelib/pyshell.py", line 1246, in runit
    self.interp.runsource(line)
  File "/home/serhiy/py/cpython3.6/Lib/idlelib/pyshell.py", line 684, in runsource
    return InteractiveInterpreter.runsource(self, source, filename)
  File "/home/serhiy/py/cpython3.6/Lib/code.py", line 64, in runsource
    code = self.compile(source, filename, symbol)
  File "/home/serhiy/py/cpython3.6/Lib/codeop.py", line 168, in __call__
    return _maybe_compile(self.compiler, source, filename, symbol)
  File "/home/serhiy/py/cpython3.6/Lib/codeop.py", line 82, in _maybe_compile
    code = compiler(source, filename, symbol)
  File "/home/serhiy/py/cpython3.6/Lib/codeop.py", line 133, in __call__
    codeob = compile(source, filename, symbol, self.flags, 1)
SyntaxWarning: assertion is always true, perhaps remove parentheses?


But other syntax warnings are converted to errors in IDLE (reported by Raymond in https://bugs.python.org/issue15248#msg349375).

In the regular interactive interpreter:

>>> 0 is 0
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True
>>> data = [
...          (1, 2, 3) # oops, missing comma!
...          (4, 5, 6)
...      ]
<stdin>:2: SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: 'tuple' object is not callable

In IDLE:

>>> 0 is 0
SyntaxError: "is" with a literal. Did you mean "=="?
>>> data = [
         (1, 2, 3) # oops, missing comma!
         (4, 5, 6)
     ]

SyntaxError: 'tuple' object is not callable; perhaps you missed a comma?
msg349417 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-11 23:22
See: https://bugs.python.org/msg349375

Because new SyntaxWarnings were added in 3.8, I expect this problem to become more prevalent.  Hopefully, this gets fixed for the 3.8 release.
msg349431 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-12 03:21
Treating a SyntaxWarning as an error is intentional, since 2001 in a Kurt Kaiser patch.  IDLE's ModifiedInterpreter subclasses code.InteractiveInterpreter.  The subclass .runsource turns a SyntaxWarning into an error, before calling the superclass method, with

    warnings.filterwarnings(action="error", category=SyntaxWarning)

As long as this is true, it should be documented.  It is included in #37825.

In interactive python (as opposed to IDLE), code is executed after a SyntaxWarning.  In the non-callable call examples, minimally "()()", "[]()", "1()", and "''()", the result is a traceback that repeats the warning message. I think the repetition is mostly noise and is better omitted. I don't know what SyntaxWarnings were present in 2001, but I presume that Kurt had the same impression, whatever it or they were.

The non-callable calls violate the 'ideal' Python grammar, which would have a production like "call = possible_callable '(', args, ')'.  Violations are caught by the compiler rather than the parser.  But they could plausibly be called a (subclass of) SyntaxError and execution avoided everywhere.

The situation is quite different with "literal is (not) something", where 'literal' includes constant tuples.  The expression is valid, normally runs without exception, and should be executed. To me, this is not a syntax issue, so SyntaxWarning seems wrong.  But as long as is used for this situation, the conversion to error must cease.  (If I knew how, I would prefer to still not call known non-callables in Shell.)

There are two subcases with different reasons for pinging users.

1. The result is obvious ("0 is ''", "0 is ()") and unchanged when 'is' is replaced with '=='.  The advice is useless.  It should instead, it seems to me, be to use True or False.  I suspect that this case is rare.

2. The result using 'is' is implementation dependent ("0 is 0", "0 is a", "1000 is 1000"). For production code, the advice to change 'is' to '==' is correct. For exploration of 'is' in a shell, which many beginners like to do, it likely is not, at least not immediately.
--

My initial messsage above is moot since compile("assert (0, 'bad')", '', 'single') no longer emit a SyntaxWarning, and the new warnings do not show the same behavior, even for similarly legal code such as "0 is 0".

My second message, about when a SyntaxWarning for a file is displayed, is still relevant, and I agree that it is a higher priority than before.  

But it is more complex than reported above.  First, there are other warnings than SyntaxWarning, and we might decide to optionally enable them.  Second, Run Module does Check Module first.  If Check Module is selected directly, there is no restart line.  Third, there will be no restart line if there is a SyntaxError, or a Tabnanny but there might be prior warnings.  I think I know how to deal with all of the above.

Module SyntaxErrors and Tabnanny errors are displayed in a popup error box, with the cursor moved to the line and maybe column of the error.  Warnings could be put in a separate box unless there is a restart.  I would prefer to not shift focus to Shell unless and until a module is run.
msg349434 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-08-12 04:17
Terry, thanks for the thorough analysis and explanation.
msg349845 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-16 05:23
I moved leaving SyntaxWarnings as warnings to #37824, as that is mostly about editing pyshell.

Changing the ordering of warnings and RESTART involves run.show_warnings.  I believe the latter should collect lines in a warnings list and print
''.join(warnings) at the appropriate time.
History
Date User Action Args
2019-08-16 05:23:48terry.reedysetmessages: + msg349845
title: IDLE: SyntaxWarning not handled properly -> IDLE: Module warnings misplaced.
2019-08-12 04:17:19rhettingersetmessages: + msg349434
2019-08-12 03:21:28terry.reedysetnosy: + taleinat
2019-08-12 03:21:15terry.reedysetpriority: high -> normal

messages: + msg349431
2019-08-11 23:22:32rhettingersetpriority: normal -> high

messages: + msg349417
2019-08-11 16:22:13serhiy.storchakasetnosy: + rhettinger
2019-08-11 16:21:42serhiy.storchakasetnosy: + serhiy.storchaka

messages: + msg349394
versions: + Python 3.9
2019-01-24 19:20:34terry.reedysetmessages: + msg334316
2018-12-11 22:33:43terry.reedysetversions: - Python 3.6
2018-09-30 20:10:30terry.reedycreate