classification
Title: IDLE: Handle Shell input warnings properly.
Type: behavior Stage: patch review
Components: IDLE Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: miss-islington, rhettinger, serhiy.storchaka, taleinat, terry.reedy
Priority: normal Keywords: patch

Created on 2019-08-11 16:32 by serhiy.storchaka, last changed 2019-08-26 06:53 by miss-islington.

Pull Requests
URL Status Linked Edit
PR 15311 open terry.reedy, 2019-08-16 05:42
PR 15500 merged terry.reedy, 2019-08-26 04:44
PR 15504 merged miss-islington, 2019-08-26 06:28
PR 15505 merged miss-islington, 2019-08-26 06:28
Messages (11)
msg349396 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-08-11 16:32
DeprecationWarning at compile time is output multiple times to the stderr (on console from which IDLE was ran), but not to IDLE Shell window.

For example, when enter '\e':

$ ./python -m idlelib

Warning (from warnings module):
  File "<pyshell#0>", line 1
    '\e'
DeprecationWarning: invalid escape sequence \e
>>> 
Warning (from warnings module):
  File "<pyshell#0>", line 1
    '\e'
DeprecationWarning: invalid escape sequence \e
>>> 
Warning (from warnings module):
  File "<pyshell#0>", line 1
    '\e'
DeprecationWarning: invalid escape sequence \e

And when close IDLE, additional output is printed in 3.8+:

>>> Exception ignored in: <idlelib.run.PseudoInputFile object at 0x7f9e3b4a15a0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/idlelib/run.py", line 488, in close
  File "/home/serhiy/py/cpython/Lib/idlelib/pyshell.py", line 1019, in close
  File "/home/serhiy/py/cpython/Lib/idlelib/editor.py", line 1058, in close
  File "/home/serhiy/py/cpython/Lib/idlelib/outwin.py", line 94, in maybesave
  File "/home/serhiy/py/cpython/Lib/idlelib/editor.py", line 991, in get_saved
AttributeError: 'NoneType' object has no attribute 'get_saved'
msg349441 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-12 06:58
None.get_saved is #35623.  I believe it is related to #35379, where I requested your comment on what I found so far.
---

1. Compiler warnings for shell input go to sys.__stderr, which only exists when IDLE is started from a console.  This will include SyntaxWarnings when not turned into errors.  pyshell 67+, warning_stream and idle_showwarning.  I have no idea why.  Compiler warnings for editor contents and runtime warnings go to shell.  For now, all should go to Shell. 

2. Warnings are formatted by run 42, idle_formatwarning.  Someone must have decided that "<stdin>:1: DeprecationWarning: invalid escape sequence \e" is newbie unfriendly.  When printed in an editor-derived window, the IDLE format will enable context menu 'goto file/line'.  But 4 lines is a lot and the top line only serves to imitate the 'Traceback ...' line.  I will think about this.  Perhaps

DeprecationWarning: invalid escape sequence \e
  File "<pyshell#1>", line 1
    '\e'

3. Warnings (at least this one) for shell input are printed 3 times.  I do not know why at present.  ModifiedInterpreter.runsource calls code.InteractiveInterpreter(self, source, filename (pyshell 669).  Perhaps the latter is buggy in somehow calling compile 3 times.  I will look another time.
msg349846 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-16 05:50
I am combining the trivial 'leave Shell input SyntaxWarnings alone (instead of making them SyntaxErrors)' part of #34857 with this issue, 'print Shell input (and the very rare internal IDLE Warnings occurring after Shell exists) in Shell, and changing the title accordingly.

The PR is a WIP with at least two bugs. Help wanted.
1. the triple output.  (Before the patch, when I started IDLE from command prompt, I only saw one.
2. Warnings are inserted before the text generating the warning.  I interpret this as intercepted \n causing compilation and printing of the warning before the \n is inserted at the end of the line and the iomark moved to the beginning of the next.  I don't know what moves the iomark.
msg349858 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-16 13:09
Il take a look at this. I've very recently gone through the relevant parts of the code and I should be able to figure it it efficiently.
msg349869 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-16 15:02
The triple (or double, see below) printing seems like a pure bug to be fixed.  A fix could be a separate PR to be merged 'immediately'.

But there is a complication with insertion location.  My bug claim was based on a one line statement, where 'before' looks wrong and 'after', as in REPL, right.

>>> <IDLE>
Warning (from warnings module):  # Printed 3 times, 2 deleted.
  File "<pyshell#0>", line 1
    0 is 0
SyntaxWarning: "is" with a literal. Did you mean "=="?
0 is 0
True

>>> 0 is 0  # REPL
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True

But now consider a more realistic compound statement.

>>> if 0 is 0:  # REPL
...   print('true')
...
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
true

With IDLE, a warning is printed *before* the user enters a blank line to signal end of statement.

>>> 
Warning (from warnings module):  # Only printed TWICE!.
  File "<pyshell#2>", line 1
    if 0 is 0:
SyntaxWarning: "is" with a literal. Did you mean "=="?
if 0 is 0:
	print('true')
	|< cursor location

A user could edit the condition, add more to the suite, or hit return. In the latter 2 cases, the warnings are repeated.  In this case, the insertion seems like the best that can be done.

Perhaps the msg349845 comment about run.show_warnings collecting warnings for possible delayed display applies to pyshell.show_warnings.  Or the warning could be put in a popup, though I do not especially like the idea of that.  Or it could be logged to a separate Warnings window, and a red "Warning logged' flashed on the status bar.

I suspect that this case is why a fake prompt was previously tacked on to the end of the warning, though I also suspect these complications are why shell input warnings were relegated to a possibly non-existent console (where the fake prompt is nonsense).

Behavior is the same with '\e' and DeprecationWarning.

This example further convinces me that multiline shell input should be isolated from both prompts and intermixed outputs.  I plan to open that issue later today.
msg349884 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-16 20:00
I see none of this on latest master (0567786d26348aa7eaf0ab1b5d038fdabe409d92) on Windows 10.  Can you specify on what OS and Python branch/version this happens?
msg349896 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-17 05:40
My first post described master at the time, when SyntaxWarnings were converted to SyntexErrors and shell input warnings, in particular, Deprecation warnings were sent to the console or nowhere.  My third post describes the situation with the PR, where SyntaxWarnings are left as warnings and shell input warnings go to shell.  An improvement, but still buggy.  I just reconfirmed after updating, compiling Python, and checking out my 'warn' branch.
msg350483 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-26 04:48
I pulled the part of PR 15311 that works, stop turning Shell SyntaxWarnings into SyntaxErrors, into PR-15500, to get it into b4.  I will update the former after merging the latter.
msg350487 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-26 06:27
New changeset 1039f39c9c6edb4c185856c19316d3a4eb561c38 by Terry Jan Reedy in branch 'master':
bpo-37824: Properly handle user input warnings in IDLE shell. (GH-15500)
https://github.com/python/cpython/commit/1039f39c9c6edb4c185856c19316d3a4eb561c38
msg350492 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-26 06:52
New changeset 077887059a5b3d38161dfd74b160c701445a1ef0 by Terry Jan Reedy (Miss Islington (bot)) in branch '3.8':
bpo-37824: Properly handle user input warnings in IDLE shell. (GH-15500)
https://github.com/python/cpython/commit/077887059a5b3d38161dfd74b160c701445a1ef0
msg350494 - (view) Author: miss-islington (miss-islington) Date: 2019-08-26 06:53
New changeset 1b15914ebb46e549ff0965c789ef03b4bac2a890 by Miss Islington (bot) in branch '3.7':
bpo-37824: Properly handle user input warnings in IDLE shell. (GH-15500)
https://github.com/python/cpython/commit/1b15914ebb46e549ff0965c789ef03b4bac2a890
History
Date User Action Args
2019-08-26 06:53:41miss-islingtonsetnosy: + miss-islington
messages: + msg350494
2019-08-26 06:52:49terry.reedysetmessages: + msg350492
2019-08-26 06:28:14miss-islingtonsetpull_requests: + pull_request15191
2019-08-26 06:28:08miss-islingtonsetstage: patch review
pull_requests: + pull_request15190
2019-08-26 06:27:59terry.reedysetmessages: + msg350487
2019-08-26 04:48:21terry.reedysetmessages: + msg350483
stage: patch review -> (no value)
2019-08-26 04:44:19terry.reedysetstage: patch review
pull_requests: + pull_request15189
2019-08-17 05:40:06terry.reedysetmessages: + msg349896
2019-08-16 20:00:13taleinatsetmessages: + msg349884
2019-08-16 15:02:19terry.reedysetmessages: + msg349869
2019-08-16 13:09:26taleinatsetmessages: + msg349858
2019-08-16 05:50:47terry.reedysetmessages: + msg349846
stage: patch review -> (no value)
2019-08-16 05:42:25terry.reedysetkeywords: + patch
stage: patch review
pull_requests: + pull_request15030
2019-08-16 04:43:40terry.reedysetnosy: + rhettinger, taleinat

title: IDLE: DeprecationWarning not handled properly -> IDLE: Handle Shell input warnings properly.
2019-08-12 06:58:00terry.reedysetmessages: + msg349441
2019-08-11 16:32:12serhiy.storchakacreate