classification
Title: build.bat should locate msbuild.exe rather than vcvarsall.bat
Type: compile error Stage: resolved
Components: Build, Windows Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: Paxxi, jkloth, ned.deily, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-06-16 22:03 by steve.dower, last changed 2017-07-08 04:51 by ned.deily. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2252 merged steve.dower, 2017-06-16 22:04
PR 2280 merged steve.dower, 2017-06-19 17:45
PR 2281 merged steve.dower, 2017-06-19 17:46
Messages (12)
msg296218 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-16 22:03
Due to how installation of Visual Studio 2017 has changed, it's very possible to end up with a valid install of the build tools without vcvarsall.bat being valid.

We only require MSBuild.exe to build, so we should find that tool and invoke it, rather than trying to configure the entire environment.
msg296362 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-19 17:34
New changeset 40a23e88994aca92c83c8e84ab8b8cdc11d7ec54 by Steve Dower in branch 'master':
bpo-30687: Fixes build scripts to find msbuild.exe and stop relying on vcvarsall.bat (#2252)
https://github.com/python/cpython/commit/40a23e88994aca92c83c8e84ab8b8cdc11d7ec54
msg296363 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-19 18:12
New changeset 06d6e3d0bb5b8a3d3105289034953a8014356a0b by Steve Dower in branch '3.6':
bpo-30687: Fixes build scripts to find msbuild.exe and stop relying on vcvarsall.bat (#2252) (#2280)
https://github.com/python/cpython/commit/06d6e3d0bb5b8a3d3105289034953a8014356a0b
msg296368 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-19 18:32
New changeset 2c899ccffda92a7f3d4e7a01f14a666504db07b5 by Steve Dower in branch '3.5':
bpo-30687: Fixes build scripts to find msbuild.exe and stop relying on vcvarsall.bat (#2252) (#2281)
https://github.com/python/cpython/commit/2c899ccffda92a7f3d4e7a01f14a666504db07b5
msg296369 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-19 18:33
Ned - this has been merged into 3.6 branch and the buildbots look good.

Feel free to move the NEWS item around and/or cherrypick into 3.6.2 at your leisure.
msg296780 - (view) Author: Pär Björklund (Paxxi) * Date: 2017-06-24 18:58
This change causes build failures when VS2017 is installed without the C++ tooling as it finds MSBuild belonging to VS2017 but it can't build using it.

Microsoft recommends using this tool https://github.com/microsoft/vswhere to find and set up the paths, it also works with VS2015.
msg296783 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-24 19:21
You're going to get build failures without the C++ tooling anyway, and msbuild should find previous installs that are compatible.

What is your system setup that causes this failure when you expect success?
msg296784 - (view) Author: Pär Björklund (Paxxi) * Date: 2017-06-24 19:26
Currently I have VS2017 installed without C++ tooling for .NET development. The C++ tooling breaks other projects I'm working on.

I have VS2015 Community update 3 installed with C++ tooling and the latest compatible Windows SDK.

Before this patch everything builds as expected using PCBuild\build.bat -e. After this patch it fails instantly with

Using "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\\MSBuild\15.0\Bin\msbuild.exe" (found in the Visual Studio 2017 registry)

C:\code\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\\MSBuild\15.0\Bin\msbuild.exe" "C:\code\cpython\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /p:Configuration=Release /p:Platform=Win32 /p:IncludeExternals=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"
C:\code\cpython\PCbuild\pythoncore.vcxproj(42,3): error MSB4019: The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Default.props" was not found. Confirm that the p ath in the <Import> declaration is correct, and that the file exists on disk.


Commenting out the following lines from find_msbuild.bat and everything is back to working

@rem VS 2017 sets exactly one install as the "main" install, so we may find MSBuild in there.
@reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\SxS\VS7" /v 15.0 /reg:32 >nul 2>nul
@if NOT ERRORLEVEL 1 @for /F "tokens=1,2*" %%i in ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\SxS\VS7" /v 15.0 /reg:32') DO @(
    @if "%%i"=="15.0" @if exist "%%k\MSBuild\15.0\Bin\msbuild.exe" @(set MSBUILD="%%k\MSBuild\15.0\Bin\msbuild.exe")
)
@if exist %MSBUILD% (set _Py_MSBuild_Source=Visual Studio 2017 registry) & goto :found
msg296791 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-24 22:30
I suspect that should be filed as a bug against VS 2017 - I'll do that when I get to work on Monday.

Our workaround should be to check for that targets file and keep searching if it's not found.

Your best workaround is to start the build from the VS 2015 developer command prompt, which will have the right MSBuild on PATH and will bypass the search.
msg296826 - (view) Author: Pär Björklund (Paxxi) * Date: 2017-06-25 15:51
I don't believe that this is a bug in Visual Studio as MSBuild is used for .NET projects as well it should be available even without the C++ tooling installed.

Checking for the targets file seems like a workable solution.
msg296838 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-06-25 21:52
The bug is  MSBuild 15.0 should be able to locate earlier versions of VC, even if you haven't installed it for VS 2017.
msg297935 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-07-08 04:51
New changeset 00134f64d982561750f57f1a0bb7bb073f9f237c by Ned Deily (Steve Dower) in branch '3.6':
bpo-30687: Fixes build scripts to find msbuild.exe and stop relying on vcvarsall.bat (#2252) (#2280)
https://github.com/python/cpython/commit/00134f64d982561750f57f1a0bb7bb073f9f237c
History
Date User Action Args
2017-07-08 04:51:40ned.deilysetmessages: + msg297935
2017-06-25 21:52:16steve.dowersetmessages: + msg296838
2017-06-25 15:51:04Paxxisetmessages: + msg296826
2017-06-24 22:30:13steve.dowersetmessages: + msg296791
2017-06-24 19:26:33Paxxisetmessages: + msg296784
2017-06-24 19:21:16steve.dowersetmessages: + msg296783
2017-06-24 18:58:42Paxxisetnosy: + Paxxi
messages: + msg296780
2017-06-19 18:34:01steve.dowersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-06-19 18:33:49steve.dowersetnosy: + ned.deily
messages: + msg296369
2017-06-19 18:32:36steve.dowersetmessages: + msg296368
2017-06-19 18:12:59steve.dowersetmessages: + msg296363
2017-06-19 17:46:27steve.dowersetpull_requests: + pull_request2330
2017-06-19 17:45:22steve.dowersetpull_requests: + pull_request2329
2017-06-19 17:34:28steve.dowersetmessages: + msg296362
2017-06-17 14:04:38jklothsetnosy: + jkloth
2017-06-16 22:04:03steve.dowersetpull_requests: + pull_request2302
2017-06-16 22:03:49steve.dowercreate