classification
Title: Build windows binaries with MS VS2019 16.8+ / MSVC 19.28+
Type: enhancement Stage: resolved
Components: Windows Versions: Python 3.10
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: h-vetinari, paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2020-11-16 21:36 by h-vetinari, last changed 2021-04-02 18:33 by steve.dower. This issue is now closed.

Messages (6)
msg381167 - (view) Author: h-vetinari (h-vetinari) Date: 2020-11-16 21:36
While Visual Studio 16.8 (<-> MSVC 19.28) has _just_ been released, I think it would be worthwhile to consider upgrading the compiler toolchain that's used to build the CPython windows binaries, particularly before the release of 3.10.

That's because many libraries (e.g. numpy/scipy) are stuck to the same compilers as CPython for ABI-compatibility, and generally, MSVC is by far the lowest common denominator in terms of C/C++ compliance, cf. https://github.com/scipy/scipy/blob/master/doc/source/toolchain.rst

For example, dropping python 3.6 support in scipy should finally enable them to use C++14/C++17, since python 3.7+ is built with Visual Studio 15.7, which has essentially complete support, cf. https://en.cppreference.com/w/cpp/compiler_support & https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering.

However (& as far as I can tell), the windows compiler version for CPython hasn't moved since the release of 3.7, cf. https://pythondev.readthedocs.io/windows.html#python-and-visual-studio-version-matrix (I know that's not an official page, but vstinner can hardly be considered a questionable source), and every release without upgrading the toolchain means another year of waiting for the ecosystem to unlock more modern C/C++.

The reason why Visual Studio 16.8 is particularly interesting, is that MS has for a very long time not paid attention to C compliance, and only recently completed C99 support, with C11/C17 following in 16.8 (though as of yet without optional aspects of the standard like atomics, threading, VLAs, complex types, etc.), cf. https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/.

Looking at the table from https://github.com/scipy/scipy/blob/master/doc/source/toolchain.rst, it would be cool if we could add the last line as follows
===================   ==============   ===================
CPython               MS Visual C++    C Standard
===================   ==============   ===================
2.7, 3.0, 3.1, 3.2       9.0           C90
3.3, 3.4                10.0           C90 & some of C99
3.5, 3.6                14.0           C90 & most of C99
3.7                     15.7           C90 & most of C99
3.8                     15.7           C90 & most of C99
3.9                     15.7           C90 & most of C99
3.10                    16.8           C99, C11*, C17
===================   ==============   ===================
* [comment about lack of C11 optionals]
msg385878 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-01-28 19:21
Judging from the link you posted to version numbering
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering
the first line should have 'MSVC 14.28' the middle column title should be  'MS Visual Studio'.

Mark Shannon today posted "Why aren't we allowing the use of C11?" on pydev.
https://mail.python.org/archives/list/python-dev@python.org/thread/PLXETSQE7PRFXBXN2QY6VNPKUTM6I7OD/
msg385904 - (view) Author: h-vetinari (h-vetinari) Date: 2021-01-29 10:15
Hey Terry

I had asked about this on discuss (https://discuss.python.org/t/toolchain-upgrade-on-windows/6377/2), and Steve provided some very valuable input.

In particular, building with the newer VS (that supports C11) should stay ABI-compatible with everything that has been built on Visual Studio 2015, 2017 and 2019:
> This is different from all previous Visual C++ versions, as they each had their own distinct runtime files, not shared with other versions.

(from https://docs.microsoft.com/en-gb/cpp/windows/universal-crt-deployment?view=msvc-160&viewFallbackFrom=vs-2019), due to the way the (now-)universal runtime is designed.

Thanks
msg385917 - (view) Author: h-vetinari (h-vetinari) Date: 2021-01-29 16:28
PS.
> Judging from the link you posted to version numbering
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering the first line should have 'MSVC 14.28' the middle column title should be 'MS Visual Studio'.

The wiki page was refactored quite extensively it seems, this is what I had been referring to: https://en.wikipedia.org/w/index.php?title=Microsoft_Visual_C%2B%2B&oldid=997067123. But I'll happily admit that I don't understand the reasons behind (the differences between) the various version numbers: MSVC++, _MSC_VER, etc.
msg385918 - (view) Author: h-vetinari (h-vetinari) Date: 2021-01-29 16:31
PPS. Also, the compiler implementation reference uses 19.x for MSVC: https://en.cppreference.com/w/cpp/compiler_support, which was the link I was trying to make, now that I'm looking at it.
msg390086 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-04-02 18:33
Closing this as not a bug, since the toolchain used to build CPython itself has no impact on the toolchain that extensions can use.

If your build tool is not detecting newer versions of MSVC, you'll need to file that against whichever backend you are using.
History
Date User Action Args
2021-04-02 18:33:56steve.dowersetstatus: open -> closed
resolution: not a bug
messages: + msg390086

stage: resolved
2021-01-29 16:31:47h-vetinarisetmessages: + msg385918
2021-01-29 16:28:50h-vetinarisetmessages: + msg385917
2021-01-29 10:15:06h-vetinarisetmessages: + msg385904
2021-01-28 19:21:09terry.reedysetnosy: + terry.reedy
messages: + msg385878
2020-11-16 21:36:20h-vetinaricreate