This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: hide command prompt when using subprocess.Popen with shell=False on Windows
Type: enhancement Stage: patch review
Components: Library (Lib), Windows Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, iMath, mmaswg, ohno, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: easy, patch

Created on 2017-04-16 17:26 by iMath, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
test.py mmaswg, 2020-03-01 09:35 Script to test if subprocess spawns additional windows
Pull Requests
URL Status Linked Edit
PR 3013 closed jeffreyrack, 2017-08-07 05:18
PR 19014 open mmaswg, 2020-03-15 10:28
Messages (30)
msg291760 - (view) Author: Philip Lee (iMath) Date: 2017-04-16 17:26
First, It is nearly useless for the command prompt to pop up during  the running time of subprocess.Popen with shell=False.
Second, the popping up command prompt would interrupt users and do bad to user experience of GUI applications.
Third, I found QProcess within Qt won't pop up the command prompt  in using.

It would be convenient to add an argument to suppress the command prompt from popping up when using subprocess.Popen with shell=False on Windows, many users are missing the feature and these are many similar feature request questions like the following 
http://stackoverflow.com/questions/7006238/how-do-i-hide-the-console-when-i-use-os-system-or-subprocess-call
http://stackoverflow.com/questions/1765078/how-to-avoid-console-window-with-pyw-file-containing-os-system-call/12964900#12964900
http://stackoverflow.com/questions/1016384/cross-platform-subprocess-with-hidden-window
msg291815 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2017-04-17 23:40
Hiding the console is required often enough that a keyword-only parameter for this would be useful. Other platforms could simply ignore it, in contrast to writing platform-dependent code that passes startupinfo. Note that this option would also hide the window of GUI programs that do not (intentionally) ignore SW_HIDE, such as notepad.
msg329979 - (view) Author: neel patel (ohno) Date: 2018-11-16 03:00
Can I work on this? I'm not sure of it's status, though.
msg330091 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-11-19 11:43
Looks like it's available.

It will be a new subprocess option to not create a new window. The underlying issue is that some applications always request a console, and if your own application doesn't have one then Windows will create it. Normally you would fix this in the target application, but we can offer an option to try and force any new windows to be hidden (though as Eryk points out, it may cause unexpected behaviour and so should not be the default).
msg362978 - (view) Author: Swgmma (mmaswg) * Date: 2020-02-29 12:29
I would like to pick up work on this (have started at link below), however I would like some help:

For testing purposes, could someone please provide a minimum working example of using subprocess that causes a command prompt to pop up?

I have not been able to reproduce it (using Python 3.7), though I vaguely remember encountering it in the past.

https://github.com/ammgws/cpython/commit/03b9dac9205b9090dd67170427d887a9952345d3
msg362979 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-02-29 12:35
If you use subprocess to launch any process that is marked SUBSYSTEM_CONSOLE, Windows will create a new console for it. For example, "python.exe" is such a process.

Eryk's answer in the very first StackOverflow link shows how to create a STARTUPINFO parameter with the correct settings. We just want to add a Boolean argument to subprocess.Popen.__init__() that can add that setting automatically.
msg363046 - (view) Author: Swgmma (mmaswg) * Date: 2020-03-01 09:35
Sorry, perhaps I did not word it clearly. Indeed the code to implement this looks trivial, however the part I am having trouble with is reproducing the reported issue in the first place.

For example, running the attached file from a command prompt (`python test.py`) does not result in any additional windows popping up.

Either my test case is wrong or there is something else going on.
msg363060 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-01 14:10
Try running that script with pythonw.exe instead.
msg363116 - (view) Author: Philip Lee (iMath) Date: 2020-03-02 05:14
To reproduce the reported issue, one could also test with ffmpeg.exe
msg363143 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-03-02 09:25
It'll have to be a program that cannot inherit the active console. Ffmpeg normally can, but running the first script with pythonw will make sure there is no console to inherit, and a new one will pop up for a regular Python process.
msg364227 - (view) Author: Swgmma (mmaswg) * Date: 2020-03-15 10:21
> Try running that script with pythonw.exe instead.

That did the trick. Confirmed that the changes are working as intended.

Running the test script posted earlier and adding the new `force_hide` kwarg to the subprocess call:

With `force_hide=False`, a command prompt window pops up.
With `force_hide=True`, no command prompt window pops up.
msg403897 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-14 11:47
Ping in case anyone from the windows team is available to review the PR
msg404094 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-16 16:25
Had a look. The change looks fine, just needs a NEWS entry, which I suggested as:

Adds new `force_hide` argument to :mod:`subprocess` functions. This passes ``SW_HIDE`` to the new process, which most applications will use to not display any window even if they normally would.
msg404095 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-16 16:26
That 'force_hide' in the NEWS shouldn't have single backticks. Make them ``double``, or the docs won't compile.
msg404122 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-16 23:27
The force_hide option could also force the use of a new hidden window, even if the current process has a console. That way the output is always hidden. For example:

    if force_hide or shell:
        if force_hide and not (creationflags & _winapi.DETACHED_PROCESS):
            creationflags |= _winapi.CREATE_NEW_CONSOLE
        startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
        startupinfo.wShowWindow = _winapi.SW_HIDE

One can also use CREATE_NO_WINDOW to create a console that has no window, as opposed to a hidden window. For example:

    if force_no_window:
        if creationflags & _winapi.CREATE_NEW_CONSOLE:
            raise ValueError('force_no_window cannot be used with '
                             'CREATE_NEW_CONSOLE')
        creationflags |= _winapi.CREATE_NO_WINDOW
msg404261 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-19 02:09
I implemented your first suggestion for `force_hide`.

Should we add your `force_no_window` suggestion as well?
msg404264 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-19 03:01
I was intending force_no_window as an alternative to adding force_hide, since CREATE_NO_WINDOW only affects console applications. Maybe a better name is force_hide_console.
msg405151 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-28 07:26
Just added a commit implementing `force_hide_console`.

What is the most trivial way to test it's behaviour?
msg405166 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-28 09:07
> What is the most trivial way to test it's behaviour?

With CREATE_NO_WINDOW, the child process is attached to a console that has no window. Thus calling GetConsoleWindow() in the child returns NULL without an error. OTOH, if the child has no console (e.g. DETACHED_PROCESS or executing "pythonw.exe"), then GetConsoleWindow() returns NULL with the last error set to ERROR_INVALID_HANDLE (6). For example:

    script = r'''
    import sys
    import ctypes
    kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
    kernel32.GetConsoleWindow.restype = ctypes.c_void_p
    ctypes.set_last_error(0)
    result = kernel32.GetConsoleWindow()
    status = bool(result or ctypes.get_last_error())
    sys.exit(status)
    '''
    args = [sys.executable, '-c', script]

The child's exit status will be 0 if CREATE_NO_WINDOW works as expected. Otherwise the exit status will be 1.
msg405177 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-28 11:24
For users who simply want to hide a process' window and do not have intimate knowledge of how the window is created by Windows, is it feasible to just have one option that handles all cases? Or do we have to implement both `force_hide` and `force_hide_console` and let users figure out which to use from the docs?
msg405205 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-28 17:21
> is it feasible to just have one option that handles all cases?

I'm in favor of whatever has broad support and is least surprising for users. I like CREATE_NO_WINDOW, which is restricted to console applications. The alternative is hiding the window, like what we do with shell=True. But the new option would also hide the main window of many GUI apps, which may come as a surprise and is hardly ever desired behavior.
msg405269 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-28 21:07
I'd rather only have one force_hide option that does its best to hide everything.

Yes, it's probably unintuitive for GUI apps, it might even be maliciously misused, but it does have occasional legitimate uses. And besides, an app can easily ignore SW_HIDE if it wants.

Bear in mind that the test suite should (mostly) operate without ctypes. So if you use ctypes to verify the test, make it a separate test that skips if ctypes cannot be imported, and have a test that runs it without verifying the window is actually hidden. That way we always at least _run_ the code, even if we only (usually) verify that it's actually hiding the window.
msg405285 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-29 05:47
If only force_hide is implemented, based on `wShowWindow=SW_HIDE`, please ensure that it's clearly documented that this option hides the main window (i.e. first top-level window) of any application that doesn't bypass the default behavior of the window manager.

This means a script can't simply use `force_hide=True` universally when running arbitrary commands, since hiding the main window of a GUI app is probably undesired behavior. If it's unknown in advance whether an executable is a console or GUI app, it can be determined via SHGetFileInfoW(). For example:

    import ctypes
    shell32 = ctypes.WinDLL('shell32', use_last_error=True)
    SHGFI_EXETYPE = 0x2000

    def is_gui_exe(path):
        result = shell32.SHGetFileInfoW(path, 0, None, 0, SHGFI_EXETYPE)
        # the high word is non-zero for a GUI app
        return (result >> 16) > 0

    >>> is_gui_exe(r'C:\Windows\pyw.exe')
    True
    >>> is_gui_exe(r'C:\Windows\py.exe')
    False

Regarding the effect of wShowWindow:

If the STARTUPINFO record defines wShowWindow, the value gets used as the normal show command for the first top-level window that's shown in the process. In particular, it overrides the first use (and only the first use) of the commands SW_SHOW, SW_SHOWNORMAL, and SW_SHOWDEFAULT (yes, only the first use of the latter; the documentation is wrong). This affects showing the first top-level window regardless of whether it's explicit via ShowWindow() or implicit via the WS_VISIBLE window style. For the latter, the implicit command is SW_SHOW, which one can override in the CreateWindowW() call by passing x=CW_USEDEFAULT and y as the show command to use. 

To completely ignore the STARTUPINFO wShowWindow value for the main window, an application can use an initial show command that wShowWindow doesn't override, such as SW_HIDE, SW_RESTORE (behaves like SW_SHOWNORMAL on first use), SW_MINIMIZE, or SW_MAXIMIZE. For example, for the first command use ShowWindow(hwnd, SW_HIDE). Subsequently calling ShowWindow(hwnd, SW_SHOWNORMAL) will show the window normally (i.e. active and restored) instead of using the STARTUPINFO wShowWindow command.
msg405298 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-29 10:06
What if we change `force_hide` from True/False to something like "off" | "console" | "all" (defaulting to "off")?
msg405302 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-29 11:12
There's nothing gained by complicating this API with more options.

Document it as "Passing *force_hide* as True attempts to start the application without creating or showing any windows. Some applications may ignore this request, and applications that are hidden often cannot be used or exited by users."

We don't have to specify all the nuances here, especially since we may come up with a more reliable way to do this in the future. Specifying too much detail in the documentation will prevent us from improving it without breaking stuff.
msg405307 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-29 14:25
> There's nothing gained by complicating this API with more options.

Yes, both options is too much. I suggested force_hide_console as an alternative to force_hide, not for both to be implemented. It would be the same as CreateNoWindow in .NET ProcessStartInfo [1], but ProcessStartInfo also has WindowStyle, and one or the other is ignored depending on the value of UseShellExecute. I see a lot of confusion online regarding these .NET ProcessStartInfo properties, which is something I'd like to avoid.

A benefit of force_hide based on wShowWindow=SW_HIDE is feature parity with ShellExecuteExW(), if subprocess ever supports the shell API. ShellExecuteExW() does not support CREATE_NO_WINDOW, but it supports a show-window command and CREATE_NEW_CONSOLE (i.e. the inverse of SEE_MASK_NO_CONSOLE).

---
[1] https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo?view=net-5.0
msg405309 - (view) Author: Swgmma (mmaswg) * Date: 2021-10-29 15:14
Thanks all for the guidance. Have gone back to the original `force_hide` with the suggested documentation.
msg405362 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-30 08:04
Here's an example test that calls WinAPI GetConsoleWindow() and IsWindowVisble() in a child process.

    if mswindows:
        try:
            import ctypes
        except ImportError:
            ctypes = None

    # added in Win32ProcessTestCase
    def test_force_hide(self):
        if ctypes:
            script = textwrap.dedent(r'''
                import sys, ctypes
                GetConsoleWindow = ctypes.WinDLL('kernel32').GetConsoleWindow
                IsWindowVisible = ctypes.WinDLL('user32').IsWindowVisible
                sys.exit(IsWindowVisible(GetConsoleWindow()))
            ''')
        else:
            script = 'import sys; sys.exit(0)'
        rc = subprocess.call([sys.executable, '-c', script], force_hide=True)
        self.assertEqual(rc, 0)

This test will work reliably even if the user has Windows Terminal set as the default terminal (available with recent builds of Windows and Windows Terminal Preview). The proxy does not hand off to Windows Terminal if the STARTUPINFO wShowWindow is SW_HIDE or SW_SHOWMINIMIZED, or if the process is created with the CREATE_NO_WINDOW flag. In these cases, it just creates a system conhost.exe session.
msg405576 - (view) Author: Swgmma (mmaswg) * Date: 2021-11-03 02:05
Anything else left to do?
msg406019 - (view) Author: Swgmma (mmaswg) * Date: 2021-11-09 13:15
Just squashed all the commits.
History
Date User Action Args
2022-04-11 14:58:45adminsetgithub: 74268
2021-11-09 13:15:20mmaswgsetmessages: + msg406019
2021-11-03 02:05:10mmaswgsetversions: + Python 3.11, - Python 3.8
2021-11-03 02:05:00mmaswgsetmessages: + msg405576
2021-10-30 08:04:33eryksunsetmessages: + msg405362
2021-10-29 15:14:03mmaswgsetmessages: + msg405309
2021-10-29 14:25:28eryksunsetmessages: + msg405307
2021-10-29 11:12:27steve.dowersetmessages: + msg405302
2021-10-29 10:06:37mmaswgsetmessages: + msg405298
2021-10-29 05:47:53eryksunsetmessages: + msg405285
2021-10-28 21:07:03steve.dowersetmessages: + msg405269
2021-10-28 17:21:47eryksunsetmessages: + msg405205
2021-10-28 11:24:38mmaswgsetmessages: + msg405177
2021-10-28 09:07:30eryksunsetmessages: + msg405166
2021-10-28 07:26:45mmaswgsetmessages: + msg405151
2021-10-19 03:01:04eryksunsetmessages: + msg404264
2021-10-19 02:09:17mmaswgsetmessages: + msg404261
2021-10-16 23:27:43eryksunsetmessages: + msg404122
2021-10-16 16:26:21steve.dowersetmessages: + msg404095
2021-10-16 16:25:46steve.dowersetmessages: + msg404094
2021-10-14 11:47:14mmaswgsetmessages: + msg403897
2020-03-15 11:03:00mmaswgsetpull_requests: - pull_request18362
2020-03-15 10:29:01mmaswgsetpull_requests: + pull_request18362
2020-03-15 10:28:14mmaswgsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request18361
2020-03-15 10:21:08mmaswgsetmessages: + msg364227
2020-03-02 09:25:06steve.dowersetmessages: + msg363143
2020-03-02 05:14:07iMathsetmessages: + msg363116
2020-03-01 14:10:35steve.dowersetmessages: + msg363060
2020-03-01 09:35:15mmaswgsetfiles: + test.py

messages: + msg363046
2020-02-29 12:35:11steve.dowersetmessages: + msg362979
2020-02-29 12:29:32mmaswgsetmessages: + msg362978
2020-02-29 05:46:59mmaswgsetnosy: + mmaswg
2018-11-19 11:43:20steve.dowersetmessages: + msg330091
versions: + Python 3.8, - Python 3.7
2018-11-16 03:00:40ohnosetnosy: + ohno
messages: + msg329979
2017-08-07 05:18:35jeffreyracksetpull_requests: + pull_request3046
2017-04-17 23:40:37eryksunsetcomponents: + Windows
versions: + Python 3.7
keywords: + easy
nosy: + paul.moore, tim.golden, eryksun, zach.ware, steve.dower

messages: + msg291815
stage: needs patch
2017-04-16 17:26:26iMathcreate