Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE and pythonw.exe stderr problem #57791

Open
serwy mannequin opened this issue Dec 11, 2011 · 15 comments
Open

IDLE and pythonw.exe stderr problem #57791

serwy mannequin opened this issue Dec 11, 2011 · 15 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@serwy
Copy link
Mannequin

serwy mannequin commented Dec 11, 2011

BPO 13582
Nosy @terryjreedy, @ned-deily, @asvetlov, @csabella, @tonybaloney
Files
  • idle_pyw.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = None
    created_at = <Date 2011-12-11.19:38:09.934>
    labels = ['3.8', 'expert-IDLE', 'type-bug', '3.7']
    title = 'IDLE and pythonw.exe stderr problem'
    updated_at = <Date 2019-05-06.20:50:56.008>
    user = 'https://github.com/serwy'

    bugs.python.org fields:

    activity = <Date 2019-05-06.20:50:56.008>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = False
    closed_date = None
    closer = None
    components = ['IDLE']
    creation = <Date 2011-12-11.19:38:09.934>
    creator = 'roger.serwy'
    dependencies = []
    files = ['23916']
    hgrepos = []
    issue_num = 13582
    keywords = ['patch']
    message_count = 15.0
    messages = ['149244', '149316', '149317', '149414', '149427', '149481', '192092', '192104', '192105', '192223', '192238', '192512', '212310', '341610', '341643']
    nosy_count = 6.0
    nosy_names = ['terry.reedy', 'ggenellina', 'ned.deily', 'asvetlov', 'cheryl.sabella', 'anthonypjshaw']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue13582'
    versions = ['Python 3.7', 'Python 3.8']

    @serwy
    Copy link
    Mannequin Author

    serwy mannequin commented Dec 11, 2011

    Running IDLE on Windows typically uses pythonw.exe. Unfortunately any error messages written to stderr will cause IDLE to terminate abruptly without an error message. This is due to __stderr__ == None.

    Attached is a patch against 3.3a0 for idle.pyw to redirect stderr messages to a dialog box. This allows IDLE to keep running so that the user can at least save their work before closing IDLE.

    @serwy serwy mannequin added topic-IDLE type-bug An unexpected behavior, bug, or error labels Dec 11, 2011
    @amauryfa
    Copy link
    Member

    "terminate abruptly"? I thought that print(file=None) silently returned, without printing but without an error.
    A delayed popup to display (otherwise discarded) output is a nice feature, though.

    @serwy
    Copy link
    Mannequin Author

    serwy mannequin commented Dec 12, 2011

    You can try triggering bug bpo-8900 quite simply. From a shell or an editor, press Ctrl+N and then Ctrl+O. Open a file and watch IDLE terminate abruptly.

    Also, see bpo-12274.

    If you want to play with this problem further, try adding a "raise Exception" to a constructor for one of the extensions. Running IDLE using python.exe will display a traceback in the console, but IDLE keeps running. However, IDLE won't even bring up a window when using pythonw.exe; it just fails to launch from the user's perspective.

    @serwy
    Copy link
    Mannequin Author

    serwy mannequin commented Dec 14, 2011

    Here is a list of open issues that describe IDLE suddenly crashing, which can be traced back to pythonw.exe:

    bpo-4765, bpo-5707, bpo-6257, bpo-6739, bpo-9404, bpo-9925, bpo-10365, bpo-11437, bpo-12274, bpo-12988, bpo-13052, bpo-13071, bpo-13078, bpo-13153

    This patch does not fix these errors, but at least these errors will no longer be fatal. This patch also "future-proofs" against yet to be discovered (or yet to be introduced) bugs into IDLE.

    @ned-deily
    Copy link
    Member

    IDLE.app on OS X also has the issue of stderr messages not being presented to users unless you know to look in the system log where they get written. So writing to stderr is not fatal but displaying them in a popup would be an improvement.

    @serwy
    Copy link
    Mannequin Author

    serwy mannequin commented Dec 14, 2011

    I don't have a Mac to test against. Is there anything I need to do to improve the patch?

    @terryjreedy
    Copy link
    Member

    Todd, do you have a Mac to test this on?

    This patch treat sending messages to a widget as a backup option. In 18318 I propose we make Idle a true gui app, with all messages other than 'No tkinter' handled by the gui. Console writing, when available, would then be a secondary option for those who want it. But in the meanwhile, I would like this or something like it applied.

    @rovitotv
    Copy link
    Mannequin

    rovitotv mannequin commented Jul 1, 2013

    Yes I have a Mac and I am glad to help, so I gave it a test run tonight. The first thing I did was apply the patch then I ran idle from the console like so:
    ./python.exe Lib/idlelib/idle.py

    For testing I used a simple print command to print to stderr:
    sys.stderr.write("spam/n")

    which I got the output of
    spam/n6

    I also tried to use the newer print function:
    print("fatal error", file=sys.stderr)

    Python 3.4 on the Mac behaved exactly the same way with or without the patch. I got the stderr output in the Python shell and nothing appeared in the console. With the patch applied I saw no dialog box to capture the stderr output. Maybe I didn't perform the test correctly?

    Another thing to consider is for Mac IDLE runs in a special mode via macosxSupport.py which I turn on by forcing runningAsOSXApp() to always return True. Even after setting runningAsOSXApp() to true a dialog box does not appear when writing to stderr.

    Maybe I am not testing this patch correctly? Let me know if I can do anything else to help, thanks.

    @terryjreedy
    Copy link
    Member

    Print in the user process goes to shell window. You need to stimulate (or just add) print or warn in the idle process, which normally goes to console, or nowhere. It is hard (intentionally, I am sure) to dynamically manipulate idle process code. Roger said "try adding a "raise Exception" to a constructor for one of the extensions". I would start with a warning in PyShell. See doc or test_warnings for an example warnings call.

    @devplayer
    Copy link
    Mannequin

    devplayer mannequin commented Jul 3, 2013

    I may be mistaken but I thought, as of not too long ago, that pythonw.exe is no longer needed by current versions. 2.7.5

    @terryjreedy
    Copy link
    Member

    No change. pythonw.exe is still present and in use.

    @rovitotv
    Copy link
    Mannequin

    rovitotv mannequin commented Jul 7, 2013

    Terry,
    Bottom line I can't seem to get this patch to do anything for me. Before the patch is applied IDLE seems to be handling warnings and exceptions just fine in PyShell on the Mac. I get no crash and the output matches the normal console. Here is a small test program I wrote:
    import warnings

    def fxn():
        # user warnings are not ignored by default and will cause a dump of information
        # to standard error.
        warnings.warn("User warning: Warn on purpose for IDLE", UserWarning)
    
    if __name__ == "__main__":
        fxn()
        print("the program should not terminate with the warning, but keep on running")
        a = 10 * 1000
        print(a)
    
        # exception testing each of these will stop the program
        # divide by zero
        b = 10 * (1/0)
        print(b)
        # variable not defined
        c = 4 + spam*3
        print(c)
        # can't convert 'int' object o str implicitly
        d = '2' + 2
        print(d)

    Then I wanted to make sure I was executing the patched code so I made sure I called idle.pyw (normally I wouldn't do that I would use idle.py). After I called the correct script I changed the code to force std error to ErrorNotify class around line 101:
    import sys

    ##if 0: # For testing
    ## sys.__stderr__ = None
    ## sys.stderr = None

    if sys.__stderr__ is None:
        sys.__stderr__ = ErrorNotify()
    
    if sys.stderr is None:
        sys.stderr = ErrorNotify()
    
    if sys.__stdout__ is None:
        sys.__stdout__ = ErrorNotify(devnull=True)
    
    if sys.stdout is None:
        sys.stdout = ErrorNotify(devnull=True)
    
    sys.__stderr__ = ErrorNotify()
    sys.stderr = ErrorNotify()

    I would expect after this code runs any message sent to stderr would go through the ErrorNotify class and a widget should appear with the stderr output. Even after this change I can't get the error widget to appear. I have not given up yet but I wanted to provide a status update.

    @terryjreedy
    Copy link
    Member

    Idle start up seems unnecessarily fragmented into multiple files.
    idlelib/main.py
    idlelib/idle.py
    idlelib/idlew.py
    can all be started from the command line by name with either python or pythonw or run once by import. idlelib/main.py can also be started by 'python(w) -m idlelib'.

    I checked that in Command Prompt
    C:\Programs\Python34>python lib/idlelib/idle.py
    C:\Programs\Python34>python lib/idlelib/idle.pyw
    C:\Programs\Python34>pythonw lib/idlelib/idle.py
    C:\Programs\Python34>pythonw lib/idlelib/idle.pyw
    all do the same thing except that the first two caused a new console python icon to appear on the taskbar.

    All three files contain
    import idlelib.PyShell
    idlelib.PyShell.main()
    which are equivalent to
    from idlelib.PyShell import main; main()
    PyShell can also by run by 'python(w) -m idlelip.PyShell' though I believe that is somewhat 'deprecated'.

    idle.py also has a path addition that seems obsolete. I believe the addition was intended for developing idle with installed python and a subrepository consisting of idlelib/. Subrepositories were possible with svn but are not with hg. In any case, proper development testing ultimately requires testing revised idle with current tkinter.py and compiled _tkinter.c. I suppose that the addition would still work to let someone clone the repository and run the repository Lib/.py, etc, with installed binaries, but that seems ultimately chancy.

    idle.pyw (which is also called by idle.bat) has additions for a scenario that I don't understand: idle.pyw is present and running but apparently not in idlelib, Idle is 'not installed', but PyShell and the rest of idlelib are somewhere on sys.path. There is nothing that I see as pythonw specific. I think this file should be dropped and any needed path manipulations added to idle.py.

    A single start up file (idle.py) should first import tkinter (and _tkinter) as in the patch, but in try;except. If the import fails print to stderr if there is one (a console) or use subprocess to start python in a console to display the message. Ditto for creating root. Once that succeeds, I think stderr or the idle process should be replaces unconditionally. A message box as in the patch is one possibility. An error log window is another. The latter could accumulate all non-fatal error messages to be edited and saved if the user wishes. I think the arg parsing code in PyShell that decides whether to open a Shell or an Editor should be moved to the startup file. Ditto for any other configuration stuff that precedes opening one or the other.

    @terryjreedy terryjreedy added the 3.7 (EOL) end of life label Jun 30, 2017
    @terryjreedy terryjreedy self-assigned this Jun 30, 2017
    @tonybaloney
    Copy link
    Mannequin

    tonybaloney mannequin commented May 6, 2019

    Revisiting this issue as it's still open. The original patch was for 3.3a0, there have been many changes to both IDLE and the Windows build since, including the recent IDLE entry point in the new Windows installer.

    Steve, Terry, please could you review to check if this issue is still relevant or can be closed off.

    @terryjreedy
    Copy link
    Member

    Good question. There are two issues when starting IDLE with pythonw.exe, so that sys.__stderr__ and sys.stderr are initially None.

    1. None.write is an attribute error that crashes Python. Therefore, don't do that. Instead use print. By default, and when 'file=None' is given directly or by reference, print writes to sys.stdout if it exists or gives up otherwise.

    I checked for possible (None).write code and did not find any except where commented out. These should be deleted or converted to prints. Possible None files are passed to traceback.print_exception and traceback.print_stack and these both use print, not write.

    All but two of the stderr crash issues Roger listed in msg149414 are closed, and the last two either should be or should have a different problem.

    Still relevant is

    1. Unprinted messages cannot be read by the user. So display them in a text widget.

    Open question 1: backup to print?, default display method?, or both (when possible)? Ned implies that both would be good on Mac, and maybe
    same would be true everywhere.

    Open question 2: what patch? I need to consider Roger's patch in relation to idlelib changes since, including new PseudoFiles, changes to text viewer, and new query module.

    The above is about sys in the IDLE GUI process. The user code process is started with sys.executable, and sys.stdout and sys.stderr are replaced with Pseudofiles that direct output to the idle process shell. The value of sys.__stdout__ and __stderr__ depend on sys.executable. If None, prints will go to sys.stdout and hence Shell, and writes raise AttributeError, which also appears in the Shell. If normal, prints and writes go to the console. I should check if enough of this is in the docs.

    It might be a good idea to wrap both processes in a top-level try-except to attempt to display any unexpected internal error.

    I will clean the nosy list as this issue does not involve Steve Dower and many others are long inactive.

    @terryjreedy terryjreedy added the 3.8 only security fixes label May 6, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    Status: No status
    Development

    No branches or pull requests

    3 participants