classification
Title: IDLE's close fails io is set to None on Mac
Type: behavior Stage: resolved
Components: IDLE, macOS Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: cheryl.sabella, miss-islington, ned.deily, rhettinger, serhiy.storchaka, taleinat, terry.reedy
Priority: normal Keywords: patch

Created on 2018-12-02 18:30 by rhettinger, last changed 2019-09-19 00:31 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10564 closed thatiparthy, 2019-09-14 05:40
PR 16212 merged terry.reedy, 2019-09-17 05:39
PR 16213 merged miss-islington, 2019-09-17 06:05
PR 16214 merged miss-islington, 2019-09-17 06:05
Messages (14)
msg330894 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-12-02 18:30
I'm not sure that sequence of events that causes this, but more than once I've gotten the following traceback.

Exception in Tkinter callback
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/__init__.py", line 1705, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/multicall.py", line 176, in handler
    r = l[i](event)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/filelist.py", line 54, in close_all_callback
    reply = edit.close()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/editor.py", line 1017, in close
    self._close()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/pyshell.py", line 309, in _close
    EditorWindow._close(self)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/editor.py", line 1021, in _close
    if self.io.filename:
AttributeError: 'NoneType' object has no attribute 'filename'
msg331023 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-12-04 10:00
Serhiy, this issue appears to be about an exception raised when an IDLE editor shutdown is called twice. Any thoughts would be appreciated. 

The recent #35263 and old #17822 are also intermittant unsolved None attribute errors.  #17614 (spunoff from *17613) fixed an instance of this exact error.  #22614 fixed a related error. 

For an editor, io is unconditionally set and unset in __init__ and _close with
 249:         self.io = io = self.IOBinding(self) # from iomenu.py
1026:         self.io = None
The reset to None is after the failing self.io.filename test, so it seems that failure is not possible.  There is no other occurrence of re 'io\s=\sNone' in idlelib.

There are, however, 3 explicit explicit tests of self.io.

1. editor.py, 1005-6, David Scherer in 2009, no issue or commit message.
    def maybesave(self):
        if self.io:  # should add '== None' here and 'return None' at end.
Called by close() before _close().


2. pyshell.py, Roger Serwy, #17614, 265-270
    def restore_file_breaks(self):  # part of editor initialization
        self.text.update()   # this enables setting "BREAK" tags to be visible
        if self.io is None:
            # can happen if IDLE closes due to the .update() call
            return
Triggered by editing a large file and hitting Alt-F4 after the editor window appears but before initialization is complete.  (Should add issue to comment.)

3. pyshell.py, Serhiy Storchaka, 154-8
    def color_breakpoint_text(self, color=True):
        "Turn colorizing of breakpoint text on or off"
        if self.io is None:
            # possible due to update in restore_file_breaks
            return

The latter two are a race condition of close being called by an event handler while initialization is ongoing.  This does not obviously apply to maybesave or this issue, but maybe somehow close can be called again while still executing.

The failing line is followed by
            self.update_recent_files_list(new_file=self.io.filename)
If there is no io, this cannot execute, and the test might be changed to
         if self.io in not None and self.io.filename:
However, self.io.close() would then fail.  Even with that fixed, I imagine that other shutdown  calls in _close() could fail.  If 'self.per = None' or 'self.top.destroy() are called once, a subsequent self.per or self.top will fail.
msg352406 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-14 04:55
The get_saved tracebacks reported in #35263 and #38128 are the same except for line numbers. The more recent, from 3 days ago on Mac.
Exception in Tkinter callback
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
    return self.func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/multicall.py", line 176, in handler
    r = l[i](event)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/filelist.py", line 54, in close_all_callback
    reply = edit.close()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/pyshell.py", line 1008, in close
    return EditorWindow.close(self)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1077, in close
    reply = self.maybesave()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/outwin.py", line 94, in maybesave
    return 'yes' if self.get_saved() else 'no'
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1010, in get_saved
    return self.undo.get_saved()
AttributeError: 'NoneType' object has no attribute 'get_saved'

None.filename from a day ago is also the same except for line numbers.  pyshell line 1008 is in PyShell(OutputWindow(EditorWindow)) so get_saved must result from closing Shell.  pyshell line 309 is in PyShellEditorWindow(EditorWindow) so must come from closing an editor window.
msg352409 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-14 06:07
Raymond, I presume 1. your tracebacks are on macOS. 2. They occur with installed, not repository Python. 3. The missing attribute depends on whether Shell or an editor window is closed either only or last.  Please verify.  What macOS version?  python.org installer?

I cannot, at least now, reproduce this on my Macbook with macOS Mohave (? the latest version) running 3.7.4 ('python3') from Terminal.  What could be different your system.  The macOS version?  Were your 3.8 reports based on the last beta (.0b4)?

I also cannot reproduce on Win10 running installed 3.7.4 or 3.8.0b4 from Command Prompt with the py launcher, I do not have any problems when closing.  
---

When running from repository 3.8 or 3.9 (master), but not 3.7, is see the following if a Shell is closed, last or not.

Exception ignored in: <idlelib.run.PseudoInputFile object at 0x05681418>
Traceback (most recent call last):
  File "f:\dev\38\lib\idlelib\run.py", line 488, in close
  File "f:\dev\38\lib\idlelib\pyshell.py", line 1020, in close
  File "f:\dev\38\lib\idlelib\editor.py", line 1062, in close
  File "f:\dev\38\lib\idlelib\outwin.py", line 94, in maybesave
  File "f:\dev\38\lib\idlelib\editor.py", line 995, in get_saved
AttributeError: 'NoneType' object has no attribute 'get_saved'

The get_saved fix proposed in PR 10564 (object.method() => object and object.method()) will fix this also.  But I want to try changing the closing order first.

Tal and Cheryl, other people testing might be helpful.
msg352410 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-09-14 06:12
1. your tracebacks are on macOS.
Yes

2. They occur with installed, not repository Python.
python installer

 3. The missing attribute depends on whether Shell or an editor window is closed either only or last. 
I don't know the trigger event

What macOS version?
Mojave 10.14.6

python.org installer? 
https://www.python.org/ftp/python/3.8.0/python-3.8.0b4-macosx10.9.pkg

Thanks for looking at this.
msg352411 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-09-14 06:13
Also, the way I start IDLE is from a terminal session:

   $ python3.8 -m idlelib.idle
msg352417 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-14 06:48
What happens if you immediately close the Shell window?  Perhaps your bug only appears after some amount of activity.  (This is not true of the one I see with repository python.)
msg352427 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-09-14 12:46
I've also seen these occasionally, but I haven't yet found a way to reproduce consistently.

Regardless, IMO the shutdown close() and _close() should be the places handling shutdown-related exceptions if possible.  In all of the tracebacks posted here, as well as those in #35263 and #17822, one such method is in the call chain, and it could simply catch and handle the raised exception.

I also think these are clear improvements that are safe to include, even without understanding the full details of the sequences of events leading to the errors.
msg352599 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-17 06:04
By reading idlelib code, I determined that the error must arise from EditorWindow.close being called twice.  I confirmed this with a debug print.

EW.close is called from close_event, PyShell.close, and filelist.close_all_callback. For the AttributeError I see, msg352409, both calls come from PyShell.close (debug print) and at least the second from PseudoInputFile.close.  Since it does not happen in 3.7, in spite of code being identical, a 3.8 change in Python shutdown must be involved.

I am rather sure that a double close is also the problem on Raymond's mac, with at least the second coming from close_all_callback.  The latter is invoked by File => exit and exit() or quit().  It might be that I do not remember seeing the same traceback because I almost never exit that way.  On Windows, I often right click the IDLE taskbar icon and select 'close all windows'.  Or the binding of close_all_callback in macOSX might be involved.

As a practical matter, I am defining the bug to be fixed as printing an annoying and essentially useless traceback to a terminal used to start IDLE.  I could spend hours trying to prevent the 2nd call I see and even if I succeeded, there would still be the 2nd call I don't see, and the problem that prompted #17822.  Clean tk shutdown is tricky and there are other unresolved issues with IDLE tests.

PR 10564 uses the direct approach suggested by Tal.  It works for me however I shut down IDLE.

Setting a Boolean flag after the first call might be cleaner, but I don't know for sure that it would prevent what Raymond sees, and I want to get a fix into 3.7.5, and I am more sure that catching AttributeErrors will work for him as well.
msg352600 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-17 06:05
New changeset dfd34a9cd58e8150c324190f746de919e140abe8 by Terry Jan Reedy in branch 'master':
bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
https://github.com/python/cpython/commit/dfd34a9cd58e8150c324190f746de919e140abe8
msg352606 - (view) Author: miss-islington (miss-islington) Date: 2019-09-17 06:23
New changeset 73ccc3322f7ad0d016478ef20237bccd0a314f0a by Miss Islington (bot) in branch '3.8':
bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
https://github.com/python/cpython/commit/73ccc3322f7ad0d016478ef20237bccd0a314f0a
msg352607 - (view) Author: miss-islington (miss-islington) Date: 2019-09-17 06:24
New changeset 3d916a7b1ee29896bece4a6a37d8084fa3c1abf6 by Miss Islington (bot) in branch '3.7':
bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
https://github.com/python/cpython/commit/3d916a7b1ee29896bece4a6a37d8084fa3c1abf6
msg352609 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-17 06:29
Before closing this, I want to review #17822 and if the specific fix is superceded by this one, remove it.
msg352761 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-09-19 00:31
#17822 is not what I was thinking of, nor are #17614 or #22614, so I considered this issue done for now.
History
Date User Action Args
2019-09-19 00:31:30terry.reedysetstatus: open -> closed
resolution: fixed
messages: + msg352761

stage: commit review -> resolved
2019-09-17 06:29:35terry.reedysetmessages: + msg352609
stage: patch review -> commit review
2019-09-17 06:24:05miss-islingtonsetmessages: + msg352607
2019-09-17 06:23:15miss-islingtonsetnosy: + miss-islington
messages: + msg352606
2019-09-17 06:05:24miss-islingtonsetpull_requests: + pull_request15815
2019-09-17 06:05:17miss-islingtonsetstage: commit review -> patch review
pull_requests: + pull_request15814
2019-09-17 06:05:07terry.reedysetmessages: + msg352600
2019-09-17 06:04:03terry.reedysettype: behavior
messages: + msg352599
stage: patch review -> commit review
2019-09-17 05:39:54terry.reedysetstage: patch review
pull_requests: + pull_request15813
2019-09-14 12:46:46taleinatsetmessages: + msg352427
2019-09-14 06:48:56terry.reedysetmessages: + msg352417
2019-09-14 06:13:42rhettingersetmessages: + msg352411
2019-09-14 06:12:11rhettingersetmessages: + msg352410
2019-09-14 06:07:17terry.reedysetnosy: + taleinat, cheryl.sabella, - ronaldoussoren
title: IDLE's close fails when io.filename set to None -> IDLE's close fails io is set to None on Mac
messages: + msg352409

stage: patch review -> (no value)
2019-09-14 05:52:18terry.reedylinkissue35263 superseder
2019-09-14 05:40:15thatiparthysetkeywords: + patch
stage: patch review
pull_requests: + pull_request15743
2019-09-14 05:27:22terry.reedylinkissue38128 superseder
2019-09-14 04:55:07terry.reedysetmessages: + msg352406
2018-12-11 21:09:46terry.reedysetnosy: + ronaldoussoren, ned.deily

components: + macOS
versions: + Python 3.8
2018-12-04 10:00:56terry.reedysetnosy: + serhiy.storchaka
messages: + msg331023
2018-12-02 18:30:10rhettingercreate