This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "Building C and C++ Extensions on Windows" docs are very out-of-date
Type: Stage: patch review
Components: Documentation, Windows Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: WildCard65, docs@python, paul.moore, shreyanavigyan, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2021-04-11 10:51 by shreyanavigyan, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
patch.patch shreyanavigyan, 2021-04-23 08:12 Patch File
Pull Requests
URL Status Linked Edit
PR 25343 closed shreyanavigyan, 2021-04-11 11:28
PR 26020 open shreyanavigyan, 2021-05-10 16:43
Messages (19)
msg390773 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-11 10:51
In the context of Docs/extending/windows.rst, the command provided, for compiling the source code and linking the libraries to create a DLL, only works for Python 32-bit versions. But the documentation doesn't inform anything about that. The PR linked with this issue adds a note section with the text :-

"""

The above commands are only applicable for Python 32-bit versions.

"""

This PR is against the master branch and if accepted, it needs to be backported to all the previous supported versions.

Kindly have a review of my PR.

Thanking you,

With Regards
msg390883 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-12 18:21
Kindly have a review of my PR.
msg391052 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-14 10:06
Kindly have a review of my PR.
msg391144 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-15 17:37
Please have a review.
msg391342 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-18 18:57
Kindly have a review of my PR.
msg391460 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-20 18:04
Kindly have a review of my patch
msg391608 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-22 15:59
Ping
msg391609 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2021-04-22 16:09
Please have some patience.  Constant requests for review or pings are frankly quite annoying.  Everyone on this project is a volunteer, and as such things only get done when someone has the time and inclination to do them.  If your issue/PR has sat idle for a month or so, then it may be time to consider a ping, but a ping every two days will quickly come to be seen as spam and ignored.
msg391615 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-04-22 16:54
I agree with Zach's comment on the PR - this topic needs an overhaul.

All the information required is going to be out there somewhere, and those of us on this issue can help fill in any gaps, but someone will need to spend a bit of time on research and laying it out nicely.

This is a great contribution opportunity for someone who would rather contribute writing and research rather than just code. However, it is a tough area, and so any improvements will be pretty well scrutinised.

Good docs here will be very impactful throughout the community, so let's make sure they're good.
msg391618 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-22 17:19
Both https://docs.python.org/3/extending/windows.html#building-on-windows and https://docs.python.org/3/extending/building.html recommends using distutils. As mentioned in PEP 632 distutils will soon be deprecated. I think it'll be a good idea to change the docs to recommend using setuptools and also change the source code for setup.py to use setuptools instead of distutils.
msg391667 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-04-23 08:12
I'm attaching a patch for improving "Building C and C++ Extensions on Windows" and "Building C and C++ Extensions" docs.
msg393380 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-05-10 12:06
Ping
msg393405 - (view) Author: William Pickard (WildCard65) * Date: 2021-05-10 14:57
I'm quite familiar with MSVC's command line and I'm quite confused on what you mean "the above commands are specific to 32-bit Python"

"/LD" is available for both 32-bit and 64-bit compilations, it implies "/MT" to the compiler and "/DLL" to the linker.

"/I" is available for both 32-bit and 64-bit compilations.

Python's lib files are named exactly the same between 32-bit and 64-bit versions.

The only thing platform specific in MSVC is the compiler. Visual Studio's C/C++ build tools ships with 4 variants of MSVC:

2 32-bit versions, 1 for targeting 32-bit and the other for targeting 64-bit (32-bit native, 32-bit cross compile to 64-bit)

The same is true for the 64-bit versions (64-bit native and 64-bit cross compile to 32-bit)

Internally, these are known as: x86, x86_x64, x64, x64_x86
msg393406 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2021-05-10 15:05
Since the removal of Rietveld from the tracker, it's not easy to review a patch as a patch file.  Please open a PR instead.

As I said on the first PR, and as William says above, I don't see any indication that your assertion that some commands are 32-bit only is true.  Please provide some evidence for your claim.
msg393409 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-05-10 15:28
I still don't think the patch goes anywhere near far enough to be useful.

Most likely we should preface that section with a "we recommend using a third-party build backend for your package, such as ... and refer to packaging.python.org for up to date suggestions" and then the rest of the section shows literally the build steps to compile an extension module (so that build backend authors have a reference).

We should *not* have a section on using setuptools, or if we do, it should consist entirely of a link to the latest setuptools documentation. Their interface is not tied to our release schedule, and could change at any time, invalidating our docs (which is a *good* thing, provided we haven't documented them).
msg393412 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-05-10 15:53
Zachary:
> As I said on the first PR, and as William says above, I don't see any indication that your assertion that some commands are 32-bit only is true.  Please provide some evidence for your claim.

cl.exe by default points to the 32-bit cl.exe not the 64-bit cl.exe. And also the library used by cl.exe by default is also 32-bit. Therefore there seems to be a linker issue. And I've tested that. It's true.

Steve:
> Most likely we should preface that section with a "we recommend using a third-party build backend for your package, such as ... and refer to packaging.python.org for up to date suggestions" and then the rest of the section shows literally the build steps to compile an extension module (so that build backend authors have a reference).
We should *not* have a section on using setuptools, or if we do, it should consist entirely of a link to the latest setuptools documentation. Their interface is not tied to our release schedule, and could change at any time, invalidating our docs (which is a *good* thing, provided we haven't documented them).

I'll remove the changes in Building C and C++ Extensions. I'll also make sure no changes are referencing setuptools. I'll be opening PR very soon.
msg393413 - (view) Author: William Pickard (WildCard65) * Date: 2021-05-10 16:03
Then it appears you're using a version of the compiler that is built for building 32-bit exes/dlls, you need to use the x64 version.

For this you need to start either the x86 Cross Tools console or the x64 native console (VS 2019) or use vcvarsall.cmd/Enter-VsDevShell (PowerShell)
msg393414 - (view) Author: Shreyan Avigyan (shreyanavigyan) * Date: 2021-05-10 16:08
I'm aware of that. But anyway I'm removing it since there are ways to overcome it.
msg393415 - (view) Author: William Pickard (WildCard65) * Date: 2021-05-10 16:09
Correction: You can use either VsDevCmd.bat/Enter-VsDevShell on VS versions that provide them (2017 and 2019 are known to include it), but to get the x64 tools you need to pass command line arguments (They default to x86 native tools).

Otherwise you must use either vcvarsall.cmd and pass the appropriate command line arguments or vcvarsx86_amd64.bat/vcvars64.bat
History
Date User Action Args
2022-04-11 14:59:44adminsetgithub: 87970
2021-05-10 16:43:14shreyanavigyansetstage: needs patch -> patch review
pull_requests: + pull_request24670
2021-05-10 16:09:32WildCard65setmessages: + msg393415
2021-05-10 16:08:58shreyanavigyansetmessages: + msg393414
2021-05-10 16:03:52WildCard65setmessages: + msg393413
2021-05-10 15:53:27shreyanavigyansetmessages: + msg393412
2021-05-10 15:28:27steve.dowersetmessages: + msg393409
2021-05-10 15:05:35zach.waresetmessages: + msg393406
2021-05-10 14:57:52WildCard65setnosy: + WildCard65
messages: + msg393405
2021-05-10 12:06:19shreyanavigyansetmessages: + msg393380
2021-04-23 08:12:48shreyanavigyansetfiles: + patch.patch

messages: + msg391667
2021-04-22 17:19:38shreyanavigyansetmessages: + msg391618
2021-04-22 16:54:47steve.dowersetmessages: + msg391615
2021-04-22 16:28:51zach.waresetnosy: + paul.moore, tim.golden, steve.dower
title: Add more info about building C/C++ Extensions on Windows using MSVC -> "Building C and C++ Extensions on Windows" docs are very out-of-date
versions: + Python 3.11, - Python 3.6, Python 3.7
components: + Windows
stage: patch review -> needs patch
2021-04-22 16:09:05zach.waresetnosy: + zach.ware
messages: + msg391609
2021-04-22 15:59:23shreyanavigyansetmessages: + msg391608
2021-04-20 18:04:41shreyanavigyansetmessages: + msg391460
2021-04-18 18:57:50shreyanavigyansetmessages: + msg391342
2021-04-15 17:37:21shreyanavigyansetmessages: + msg391144
2021-04-14 10:06:37shreyanavigyansetmessages: + msg391052
2021-04-12 18:21:43shreyanavigyansetmessages: + msg390883
2021-04-11 11:28:56shreyanavigyansetkeywords: + patch
stage: patch review
pull_requests: + pull_request24077
2021-04-11 10:51:41shreyanavigyancreate