Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE's close fails io is set to None on Mac #79560

Closed
rhettinger opened this issue Dec 2, 2018 · 15 comments
Closed

IDLE's close fails io is set to None on Mac #79560

rhettinger opened this issue Dec 2, 2018 · 15 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes OS-mac topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@rhettinger
Copy link
Contributor

BPO 35379
Nosy @rhettinger, @terryjreedy, @taleinat, @ned-deily, @serhiy-storchaka, @csabella, @miss-islington
PRs
  • bpo-35379: Check IDLE objects before calling method  #10564
  • bpo-35379: When exiting IDLE, catch any AttributeError #16212
  • [3.8] bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212) #16213
  • [3.7] bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212) #16214
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2019-09-19.00:31:30.057>
    created_at = <Date 2018-12-02.18:30:09.930>
    labels = ['OS-mac', '3.8', 'expert-IDLE', 'type-bug', '3.7']
    title = "IDLE's close fails io is set to None on Mac"
    updated_at = <Date 2020-07-28.23:35:28.351>
    user = 'https://github.com/rhettinger'

    bugs.python.org fields:

    activity = <Date 2020-07-28.23:35:28.351>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2019-09-19.00:31:30.057>
    closer = 'terry.reedy'
    components = ['IDLE', 'macOS']
    creation = <Date 2018-12-02.18:30:09.930>
    creator = 'rhettinger'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35379
    keywords = ['patch']
    message_count = 15.0
    messages = ['330894', '331023', '352406', '352409', '352410', '352411', '352417', '352427', '352599', '352600', '352606', '352607', '352609', '352761', '374536']
    nosy_count = 7.0
    nosy_names = ['rhettinger', 'terry.reedy', 'taleinat', 'ned.deily', 'serhiy.storchaka', 'cheryl.sabella', 'miss-islington']
    pr_nums = ['10564', '16212', '16213', '16214']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue35379'
    versions = ['Python 3.7', 'Python 3.8']

    @rhettinger
    Copy link
    Contributor Author

    I'm not sure that sequence of events that causes this, but more than once I've gotten the following traceback.

    Exception in Tkinter callback
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/__init__.py", line 1705, in __call__
        return self.func(*args)
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/multicall.py", line 176, in handler
        r = l[i](event)
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/filelist.py", line 54, in close_all_callback
        reply = edit.close()
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/editor.py", line 1017, in close
        self._close()
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/pyshell.py", line 309, in _close
        EditorWindow._close(self)
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/editor.py", line 1021, in _close
        if self.io.filename:
    AttributeError: 'NoneType' object has no attribute 'filename'

    @terryjreedy
    Copy link
    Member

    Serhiy, this issue appears to be about an exception raised when an IDLE editor shutdown is called twice. Any thoughts would be appreciated.

    The recent bpo-35263 and old bpo-17822 are also intermittant unsolved None attribute errors. bpo-17614 (spunoff from *17613) fixed an instance of this exact error. bpo-22614 fixed a related error.

    For an editor, io is unconditionally set and unset in __init__ and _close with
    249: self.io = io = self.IOBinding(self) # from iomenu.py
    1026: self.io = None
    The reset to None is after the failing self.io.filename test, so it seems that failure is not possible. There is no other occurrence of re 'io\s=\sNone' in idlelib.

    There are, however, 3 explicit explicit tests of self.io.

    1. editor.py, 1005-6, David Scherer in 2009, no issue or commit message.
      def maybesave(self):
      if self.io: # should add '== None' here and 'return None' at end.
      Called by close() before _close().

    2. pyshell.py, Roger Serwy, bpo-17614, 265-270
      def restore_file_breaks(self): # part of editor initialization
      self.text.update() # this enables setting "BREAK" tags to be visible
      if self.io is None:
      # can happen if IDLE closes due to the .update() call
      return
      Triggered by editing a large file and hitting Alt-F4 after the editor window appears but before initialization is complete. (Should add issue to comment.)

    3. pyshell.py, Serhiy Storchaka, 154-8
      def color_breakpoint_text(self, color=True):
      "Turn colorizing of breakpoint text on or off"
      if self.io is None:
      # possible due to update in restore_file_breaks
      return

    The latter two are a race condition of close being called by an event handler while initialization is ongoing. This does not obviously apply to maybesave or this issue, but maybe somehow close can be called again while still executing.

    The failing line is followed by
    self.update_recent_files_list(new_file=self.io.filename)
    If there is no io, this cannot execute, and the test might be changed to
    if self.io in not None and self.io.filename:
    However, self.io.close() would then fail. Even with that fixed, I imagine that other shutdown calls in _close() could fail. If 'self.per = None' or 'self.top.destroy() are called once, a subsequent self.per or self.top will fail.

    @terryjreedy terryjreedy added OS-mac 3.8 only security fixes labels Dec 11, 2018
    @terryjreedy
    Copy link
    Member

    The get_saved tracebacks reported in python/cpython#79444 and python/issues-test-cpython#38128 are the same except for line numbers. The more recent, from 3 days ago on Mac.
    Exception in Tkinter callback
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
        return self.func(*args)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/multicall.py", line 176, in handler
        r = l[i](event)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/filelist.py", line 54, in close_all_callback
        reply = edit.close()
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/pyshell.py", line 1008, in close
        return EditorWindow.close(self)
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1077, in close
        reply = self.maybesave()
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/outwin.py", line 94, in maybesave
        return 'yes' if self.get_saved() else 'no'
      File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/idlelib/editor.py", line 1010, in get_saved
        return self.undo.get_saved()
    AttributeError: 'NoneType' object has no attribute 'get_saved'

    None.filename from a day ago is also the same except for line numbers. pyshell line 1008 is in PyShell(OutputWindow(EditorWindow)) so get_saved must result from closing Shell. pyshell line 309 is in PyShellEditorWindow(EditorWindow) so must come from closing an editor window.

    @terryjreedy
    Copy link
    Member

    Raymond, I presume 1. your tracebacks are on macOS. 2. They occur with installed, not repository Python. 3. The missing attribute depends on whether Shell or an editor window is closed either only or last. Please verify. What macOS version? python.org installer?

    I cannot, at least now, reproduce this on my Macbook with macOS Mohave (? the latest version) running 3.7.4 ('python3') from Terminal. What could be different your system. The macOS version? Were your 3.8 reports based on the last beta (.0b4)?

    I also cannot reproduce on Win10 running installed 3.7.4 or 3.8.0b4 from Command Prompt with the py launcher, I do not have any problems when closing.
    ---

    When running from repository 3.8 or 3.9 (master), but not 3.7, is see the following if a Shell is closed, last or not.

    Exception ignored in: <idlelib.run.PseudoInputFile object at 0x05681418>
    Traceback (most recent call last):
      File "f:\dev\38\lib\idlelib\run.py", line 488, in close
      File "f:\dev\38\lib\idlelib\pyshell.py", line 1020, in close
      File "f:\dev\38\lib\idlelib\editor.py", line 1062, in close
      File "f:\dev\38\lib\idlelib\outwin.py", line 94, in maybesave
      File "f:\dev\38\lib\idlelib\editor.py", line 995, in get_saved
    AttributeError: 'NoneType' object has no attribute 'get_saved'

    The get_saved fix proposed in PR 10564 (object.method() => object and object.method()) will fix this also. But I want to try changing the closing order first.

    Tal and Cheryl, other people testing might be helpful.

    @terryjreedy terryjreedy changed the title IDLE's close fails when io.filename set to None IDLE's close fails io is set to None on Mac Sep 14, 2019
    @rhettinger
    Copy link
    Contributor Author

    1. your tracebacks are on macOS.
      Yes

    2. They occur with installed, not repository Python.
      python installer

    3. The missing attribute depends on whether Shell or an editor window is closed either only or last.
      I don't know the trigger event

    What macOS version?
    Mojave 10.14.6

    python.org installer?
    https://www.python.org/ftp/python/3.8.0/python-3.8.0b4-macosx10.9.pkg

    Thanks for looking at this.

    @rhettinger
    Copy link
    Contributor Author

    Also, the way I start IDLE is from a terminal session:

       $ python3.8 -m idlelib.idle

    @terryjreedy
    Copy link
    Member

    What happens if you immediately close the Shell window? Perhaps your bug only appears after some amount of activity. (This is not true of the one I see with repository python.)

    @taleinat
    Copy link
    Contributor

    I've also seen these occasionally, but I haven't yet found a way to reproduce consistently.

    Regardless, IMO the shutdown close() and _close() should be the places handling shutdown-related exceptions if possible. In all of the tracebacks posted here, as well as those in bpo-35263 and bpo-17822, one such method is in the call chain, and it could simply catch and handle the raised exception.

    I also think these are clear improvements that are safe to include, even without understanding the full details of the sequences of events leading to the errors.

    @terryjreedy
    Copy link
    Member

    By reading idlelib code, I determined that the error must arise from EditorWindow.close being called twice. I confirmed this with a debug print.

    EW.close is called from close_event, PyShell.close, and filelist.close_all_callback. For the AttributeError I see, msg352409, both calls come from PyShell.close (debug print) and at least the second from PseudoInputFile.close. Since it does not happen in 3.7, in spite of code being identical, a 3.8 change in Python shutdown must be involved.

    I am rather sure that a double close is also the problem on Raymond's mac, with at least the second coming from close_all_callback. The latter is invoked by File => exit and exit() or quit(). It might be that I do not remember seeing the same traceback because I almost never exit that way. On Windows, I often right click the IDLE taskbar icon and select 'close all windows'. Or the binding of close_all_callback in macOSX might be involved.

    As a practical matter, I am defining the bug to be fixed as printing an annoying and essentially useless traceback to a terminal used to start IDLE. I could spend hours trying to prevent the 2nd call I see and even if I succeeded, there would still be the 2nd call I don't see, and the problem that prompted bpo-17822. Clean tk shutdown is tricky and there are other unresolved issues with IDLE tests.

    PR 10564 uses the direct approach suggested by Tal. It works for me however I shut down IDLE.

    Setting a Boolean flag after the first call might be cleaner, but I don't know for sure that it would prevent what Raymond sees, and I want to get a fix into 3.7.5, and I am more sure that catching AttributeErrors will work for him as well.

    @terryjreedy terryjreedy added the type-bug An unexpected behavior, bug, or error label Sep 17, 2019
    @terryjreedy
    Copy link
    Member

    New changeset dfd34a9 by Terry Jan Reedy in branch 'master':
    bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
    dfd34a9

    @miss-islington
    Copy link
    Contributor

    New changeset 73ccc33 by Miss Islington (bot) in branch '3.8':
    bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
    73ccc33

    @miss-islington
    Copy link
    Contributor

    New changeset 3d916a7 by Miss Islington (bot) in branch '3.7':
    bpo-35379: When exiting IDLE, catch any AttributeError. (GH-16212)
    3d916a7

    @terryjreedy
    Copy link
    Member

    Before closing this, I want to review bpo-17822 and if the specific fix is superceded by this one, remove it.

    @terryjreedy
    Copy link
    Member

    bpo-17822 is not what I was thinking of, nor are bpo-17614 or bpo-22614, so I considered this issue done for now.

    @terryjreedy
    Copy link
    Member

    bpo-41413 exposed another double save on Mac issue.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes OS-mac topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants