Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use of STATUS_CONTROL_C_EXIT in Modules/main.c breaks compilation with non MSC compilers #81146

Closed
erikjanss mannequin opened this issue May 19, 2019 · 14 comments
Closed
Labels
3.8 only security fixes build The build process and cross-build OS-windows

Comments

@erikjanss
Copy link
Mannequin

erikjanss mannequin commented May 19, 2019

BPO 36965
Nosy @pfmoore, @vstinner, @tjguk, @zware, @eryksun, @zooba, @erikjanss
PRs
  • bpo-36965: include windows.h to be able to use STATUS_CONTROL_C_EXIT #13421
  • [3.7] bpo-36965: replace include of crtdbg.h with windows.h, backport 925af1d 3.7 (GH-13421) #13471
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-05-22.11:05:10.336>
    created_at = <Date 2019-05-19.13:02:26.390>
    labels = ['build', '3.8', 'OS-windows']
    title = 'use of STATUS_CONTROL_C_EXIT in Modules/main.c breaks compilation with non MSC compilers'
    updated_at = <Date 2019-05-22.11:05:10.335>
    user = 'https://github.com/erikjanss'

    bugs.python.org fields:

    activity = <Date 2019-05-22.11:05:10.335>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-05-22.11:05:10.336>
    closer = 'vstinner'
    components = ['Windows']
    creation = <Date 2019-05-19.13:02:26.390>
    creator = 'erikjanss'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36965
    keywords = ['patch']
    message_count = 14.0
    messages = ['342845', '342848', '342932', '342934', '342935', '342936', '342943', '342977', '342998', '343009', '343013', '343018', '343172', '343173']
    nosy_count = 7.0
    nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'erikjanss']
    pr_nums = ['13421', '13471']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue36965'
    versions = ['Python 3.8']

    @erikjanss
    Copy link
    Mannequin Author

    erikjanss mannequin commented May 19, 2019

    in Modules/main.c STATUS_CONTROL_C_EXIT is included conditionally if compiling when _MSC_VER is defined. Later on STATUS_CONTROL_C_EXIT is used if MS_WINDOWS is defined.

    This breaks compilation of main.c with any non MSC compiler while compiling for MS Windows.

    This appears to have been introduced in #56332 for bpo-36142

    @erikjanss erikjanss mannequin added 3.8 only security fixes OS-windows build The build process and cross-build labels May 19, 2019
    @erikjanss
    Copy link
    Mannequin Author

    erikjanss mannequin commented May 19, 2019

    According to the Microsoft documentation at

    https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/using-ntstatus-values

    system-supplied status codes are defined in ntstatus.h. and the NTSTATUS type is defined in ntdef.h

    PR 13421 includes both ntstatus.h and ntdef.h to be able to use STATUS_CONTROL_C_EXIT when compiling for windows.

    @zooba
    Copy link
    Member

    zooba commented May 20, 2019

    Is including just "winnt.h" sufficient?

    It's very hard to tell which Windows headers are "public" vs "internal", but winternl.h is certainly one of the internal ones (according to the big warning comment at the top of the file)

    @erikjanss
    Copy link
    Mannequin Author

    erikjanss mannequin commented May 20, 2019

    including "winnt.h" gives me compilation problems with other undefined types.

    however, simply including "windows.h" instead of both includes compiles just fine.

    so maybe that is sufficient ??

    @zooba
    Copy link
    Member

    zooba commented May 20, 2019

    Some people say "windows.h" is the only one you're ever supposed to include, so if that works best, let's go with that :)

    @erikjanss
    Copy link
    Mannequin Author

    erikjanss mannequin commented May 20, 2019

    ok, thank you for the advice, I'll keep it in mind and adapt the PR !

    @erikjanss
    Copy link
    Mannequin Author

    erikjanss mannequin commented May 20, 2019

    PR has been changed to include "windows.h" ...

    @eryksun
    Copy link
    Contributor

    eryksun commented May 21, 2019

    "crtdbg.h" doesn't provide STATUS_CONTROL_C_EXIT, but it should be fine to remove it anyway. I think it was left behind by accident in 2007. It was added to support a PYTHONNOERRORWINDOW environment variable, but then this idea was dropped in favor of extending the msvcrt module:

    I presume STATUS_CONTROL_C_EXIT gets included from "winnt.h" -> "Windows.h" -> "Include/internal/pycore_condvar.h" -> "Include/internal/pycore_gil.h" -> "Include/internal/pycore_pystate.h".

    If [STATUS_]CONTROL_C_EXIT isn't defined, I suggest defining WIN32_LEAN_AND_MEAN before including "Windows.h". This reduces the number of included headers from about 350 down to about 200. Also, to stay strictly within the Windows API, we might want to use CONTROL_C_EXIT (from [min]winbase.h) instead of STATUS_CONTROL_C_EXIT (from "winnt.h").

    @vstinner
    Copy link
    Member

    New changeset 925af1d by Victor Stinner (Erik Janssens) in branch 'master':
    bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421)
    925af1d

    @vstinner
    Copy link
    Member

    eryksun commented there, but I prefer to discuss here:
    925af1d#commitcomment-33617265

    ""Windows.h" was already being included, as I mentioned on the issue tracker, because we certainly were not getting STATUS_CONTROL_C_EXIT from "crtdbg.h", a header that was left in this file accidentally about 12 years ago. If it's included explicitly here, also define WIN32_LEAN_AND_MEAN to cut the number of included headers by about a half."

    I prefer to explicitly include windows.h, it doesn't hurt :-)

    WIN32_LEAN_AND_MEAN is defined by Include/internal/pycore_condvar.h which is indirectly included by pycore_pystate.h:

    #include "pycore_gil.h"   /* _gil_runtime_state  */

    pycore_gil.h:

    #include "pycore_condvar.h"

    By the way, WIN32_LEAN_AND_MEAN caused me issues while working on bpo-36728:
    https://twitter.com/VictorStinner/status/1127884878027079680

    I managed to workaround the issue: commit d5d9e81

    Extract of (fixed) posixmodule.c:
    ---

    ...
    #include "Python.h"
    #ifdef MS_WINDOWS
       /* include <windows.h> early to avoid conflict with pycore_condvar.h:
    
            #define WIN32_LEAN_AND_MEAN
            #include <windows.h>
    
      FSCTL_GET_REPARSE_POINT is not exported with WIN32_LEAN_AND_MEAN. \*/
    

    # include <windows.h>
    #endif

    #include "pycore_ceval.h"     /* _PyEval_ReInitThreads() */
    #include "pycore_pystate.h"   /* _PyRuntime */
    ...

    @vstinner
    Copy link
    Member

    WIN32_LEAN_AND_MEAN is defined by Include/internal/pycore_condvar.h (...)

    It would be nice to get ride of #include <windows.h> in Python headers. The <windows.h> was introduced by this commit:

    commit 2ebc5ce
    Author: Eric Snow <ericsnowcurrently@gmail.com>
    Date: Thu Sep 7 23:51:28 2017 -0600

    bpo-30860: Consolidate stateful runtime globals. (bpo-3397)
    
    * group the (stateful) runtime globals into various topical structs
    * consolidate the topical structs under a single top-level _PyRuntimeState struct
    * add a check-c-globals.py script that helps identify runtime globals
    
    Other globals are excluded (see globals.txt and check-c-globals.py).
    

    The current problem is that we need the HANDLE type which comes from <windows.h> to get the full structure:

    typedef struct _PyCOND_T
    {
        HANDLE sem;
        int waiting; /* to allow PyCOND_SIGNAL to be a no-op */
    } PyCOND_T;

    I tried to avoid "HANDLE" using:

    typedef struct _PyCOND_T
    {
        void* sem;
        int waiting; /* to allow PyCOND_SIGNAL to be a no-op */
    } PyCOND_T;

    ... but then pycore_condvar.h compilation fails on "typedef CRITICAL_SECTION PyMUTEX_T;": CRITICAL_SECTION type is not defined :-(

    By the way, Python still uses _PY_EMULATED_WIN_CV by default, whereas we want to drop Vista support: bpo-32592.

    @eryksun
    Copy link
    Contributor

    eryksun commented May 21, 2019

    FSCTL_GET_REPARSE_POINT is not exported with WIN32_LEAN_AND_MEAN

    You can explicitly include "winioctl.h" after "windows.h". Getting it from "windows.h" is indirect via "winscard.h" (smart card services), which will be skipped if either WIN32_LEAN_AND_MEAN or NOCRYPT is defined.

    @vstinner
    Copy link
    Member

    New changeset 791e5fc by Victor Stinner (Erik Janssens) in branch '3.7':
    bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421) (GH-13471)
    791e5fc

    @vstinner
    Copy link
    Member

    The initial issue is fixed in 3.7 and master branches. If you want to work on related issue like WIN32_LEAN_AND_MEAN, please open a separated issue.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes build The build process and cross-build OS-windows
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants