In the process of eliminating compiler warnings on my buildbot, I needed to update VS2015 to the latest toolset (VS2015 Update 3).  This in turn now causes an error due about not having the required version of Windows SDK installed.

It seems that the detection logic for that uses a hard-coded list which may not be up-to-date (and possibly incorrect for some installs).  Referenced PR fixes this.

Also, my buildbot *should* fail until this change is merged.

Looks good to me. I can't apply tags from my phone, apparently, so someone else will have to merge and mark the backport versions.

Is this issue related to bpo-34977: Add Windows App Store package?

No, it just automates something that's been manual for a while. Installing VS 2017 on build bots is still a manual process ;)

Sorry Victor, I should have added more context when I nosy'ed you to this issue.

Since you seem to be the "buildbot keeper" as of late, I wanted to give you a pointer to the reason for my failing buildbot.  Which, as I surmised, is failing compile due to the missing SDK error that is fixed by this PR ;)

After I reverted bpo-34977 change, most Windows buildbots are back to green.

> In the process of eliminating compiler warnings on my buildbot, I needed to update VS2015 to the latest toolset (VS2015 Update 3).  This in turn now causes an error due about not having the required version of Windows SDK installed.

Is it a warning or an error? What is the warning/error message? What is your buildbot?

Right now, it seems like only "AMD64 Windows7 SP1 3.x" is still red:

"2>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\x64\PlatformToolsets\v140\Toolset.targets(36,5): error MSB8036: The Windows SDK version 10.0.17134.0 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". [C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pythoncore.vcxproj]"

https://buildbot.python.org/all/#/builders/40/builds/1322

Are you talking about this error?

> Is it a warning or an error? What is the warning/error message? What is your buildbot?

It is a compiler error as you point out below (with message).  By buildbot is:

https://buildbot.python.org/all/#/workers/12

> Right now, it seems like only "AMD64 Windows7 SP1 3.x" is still red:
>
> "2>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\x64\PlatformToolsets\v140\Toolset.targets(36,5): error MSB8036: The Windows SDK version 10.0.17134.0 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". [C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pythoncore.vcxproj]"
>
> https://buildbot.python.org/all/#/builders/40/builds/1322
>
> Are you talking about this error?

All the builders on my buildbot can no longer find a matching SDK
version due to how the SDK version detection is handled.  Previously,
my buildbot had VS2015.2 installed whose build tools (version 1.2)
didn't use the property `DefaultWindowsSDKVersion` to select the SDK
version and just defaulted to Windows 8.1 SDK.  Now that I've updated
to VS2015.3 (includes build tools 1.4) it *does* use this property to
select the SDK version.  However, I have a newer SDK installed than is
listed in the hard-coded list so it defaults to highest one in the
list causing the missing SDK error.

This PR changes the detection logic to simply accept what SDK the user
has installed as long as it is newer (or the same) as the minimum
allowed version.

For those versed in MSBuild, this should be fairly self-evident, but
hopefully this helps those who are not :)

Not to be impatient or anything, but this change is keeping my buildbot from being useful.  Would it be possible for someone to merge as it has already been approved (msg331263).

Did you see my comment about converting the version numbers to System.Version before comparing? (I didn't know it could be done the first time I looked :) )

Sorry for the delay, but I wanted to get an environment that still had an older VS2015 install to test against.  VS2015 prior to Update 3 use a different heuristic to determine the SDK version for building.

I've made the following changes:
- version checking uses System.Version,
- WindowsTargetPlatformVersion is set to the detected Win10 SDK (this is the offical setting anyway, but required for VS2015 prior to Update 3)
- added display of the detected SDK to ShowVersionInfo target (build.bat -V)

New changeset f46eccd0ffe65333035c3820886295b71c41ab6e by Steve Dower (Jeremy Kloth) in branch 'master':
bpo-35433: Properly detect installed SDK versions (GH-11009)
https://github.com/python/cpython/commit/f46eccd0ffe65333035c3820886295b71c41ab6e

Thanks for being thorough. I've merged the change now.

Not making this change for Python 3.6, as that apparently breaks with the latest SDK (seen in the Pipelines build). So we'll leave 10.0.15063.0 as the maximum version for that - it's about to go into security mode anyway, and we don't rely on anything from the SDK with a security impact. (It's all import libraries, and the static libraries should be coming from the compiler toolset and not the platform SDK.)

New changeset c83ec055a09137e4b24f70e3bb5f887dc0ae6e8e by Miss Islington (bot) in branch '3.7':
bpo-35433: Properly detect installed SDK versions (GH-11009)
https://github.com/python/cpython/commit/c83ec055a09137e4b24f70e3bb5f887dc0ae6e8e

I've added a PR for 3.6 using a different methodology for finding the available SDK.

Without some change, my buildbot will continue to stay in the red (for 3.6).  It does not the the highest SDK currently (previously?) listed the hard-coded list.  It has the latest (17763), the latest for VS2015 (14393) and the baseline (10586).  I would like to see a working build for VS2015 on 3.6.  Once 3.6 is wrapped, I'll update the buildbot to VS2017 as that is what is listed in the README for building 3.7+, whereas 3.6 lists VS2015.

I reopen the issue since https://github.com/python/cpython/pull/11060 for Python 3.6 is still open.

New changeset f04cc5fc0d2f644cccb57543aae487ee30091924 by Steve Dower (Jeremy Kloth) in branch '3.6':
[3.6] bpo-35433: Properly detect installed SDK versions (GH-11009)
https://github.com/python/cpython/commit/f04cc5fc0d2f644cccb57543aae487ee30091924

This commit broke the following (at least) the buildbots:

https://buildbot.python.org/all/#/builders/38/builds/751
https://buildbot.python.org/all/#/builders/31/builds/719

can someone work on a fix? Otherwise we would have to revert the commit per our policy with buildbot failures

I'm not that familiar (ok, at all) with the build process configuration, but in looking at the changes to python.props, it appears to now enforce the minimum in the build process regardless of whether it is found, whereas before the build tool was allowed to pick a default.

On my Win8/10 builders, I don't have an SDK that matches anything in the new checks (10586, 14393, 15063), but instead just have a single later (16299) version installed.  The Win7 buildbot does, however, also have an earlier 15063.

So what happens if none of the checked versions exist?  Am I correct that it'll enforce trying to use the minimum (10586) which won't work on my Win8/10 workers since it doesn't exist, while the default behavior to just use the SDK I have would be fine

Or is the build tool supposed to ignore the missing SDK and still use a default if later?

It seems to me the older default of letting the build tool apply its own defaults if no known matching SDK is found is better than specifying a possibly missing SDK, but obviously the older default caused an issue in Jeremy's original case.  And since the prior behavior was just to try to avoid the build tool from defaulting to too old an SDK, another solution to problems in that case might be to remove the oldest SDK, thus moving the default forward.

I'm also looking at it, so please don't revert it just yet.

I just installed 10.0.15063 onto my dev machine to test another 3.6 fix and it worked fine, so perhaps there's an option that is off here?

Both io.h and stddef.h are from the UCRT, which *may* go through a different detection process than the rest of the WinSDK. That might be the cause.

The "didn't find any supported SDKs" behavior is to try and use the earliest, which should fail, but all the others were going to fail too. I wonder if checking the installed UAP platform data is the wrong check, and we should look for a specific Include file instead?

Well, correct me if I'm wrong, but installing 15063 would then match one of the checks, and become the selected SDK, so be expected to work fine, right?

I think (not sure) that the issue with my Win8/10 workers is they only have the later 16299.  So in that case the build process is trying to enforce 10586 as a minimum (due to not finding anything else) which can't work.  Assuming I'm interpreting the new WindowsTargetPlatformVersion definition correctly).

When you say the "others were going to fail too" I don't think that's correct in all cases.  That is, the installed version doesn't match the checked versions, but it's later, so it could work fine.  And in this case it's the only version available so the default build tool selection is fine.

I'm guessing Jeremy's case was having older (not workable) versions and still not matching one of the checked versions, in which case the build default of oldest would fail.  And yes, in that case enforcing a missing version is no real difference since both options fail.

Not sure if solving both cases is possible, but at least letting the build tool default to an available SDK if no match is found seems more conservative.  It might still fail, but there are cases where it won't.  And it seems more likely over time that someone may only have a later SDK than that they'll have an earlier one or a guaranteed match to those being checked.  Or if we want to enforce specific versions, publishing them and forcing the build to fail hard would be an option too.

I don't think the difference in registry vs. folder checks is an issue, at least not for me.  On the Win8/10 workers, both the registry and filesystem are in agreement - the only thing available is 16299.

I'm also not sure if it's possible, but it would be great to have some output somewhere about the selection.  Right now there's nothing in the compiler stage output of a working vs. failing build to indicate why they are behaving differently.  But that's probably a separate topic.

Oh, since my reading comprehension must be low today, it appears like Jeremy actually had a closer situation previously as I'm in now, with a later (not older) version of the SDK that wasn't in the list.

Which is interesting, since he got an error about the SDK whereas my workers weren't having an issue at that point.

But I guess it still comes down to what to do if the hardcoded list doesn't match what's on the system, but the SDK on the system will actually work.  Complain or let the default get used anyway?

(sorry for the rapid updates)

I'm also fairly sure that none of my workers have update 3 for VS2015.  They do however all have VS2017 - but I think VS2015 still gets picked for 3.6 if both are present, right?  So that's another variable, in that my workers wouldn't have run into the update 3 issue with the older configuration that Jeremy did.  

Not sure if there's a way to have the build setup to work for both cases.

So before the change, the 16299 SDK wasn't being detected either, but perhaps the 10240 one was? That had some known issues in the debug UCRT, which is why the version gets printed out towards the end of the build.

The only thing I can think of would be the WindowsTargetPlatformVersion setting, which is new. It may be that it is used differently on different MSBuild versions (which roughly match to VS versions) by the built-in target files.

Are you able to run "MSBuild pythoncore.vcxproj /t:ShowVersionInfo /flp:v=diag" and upload the MSBuild.log file (or email it to me)?

> So before the change, the 16299 SDK wasn't being detected either, but perhaps the 10240 one was?

So I'm just confused....

It does seems likely that 10240 of the UCRT was being used (based on the attached msbuild logs).  Howevr, the UCRT warning in the build process was fine, and said I was using 16299 (see https://buildbot.python.org/all/#/builders/31/builds/717/steps/1/logs/stdio for the last working build on Win10).  

I'm not sure what to trust - the msbuild log or whatever the ucrtused test is doing for detection?

In either case, this issue commmit does seem to force selection of the non-existent 15063 SDK, at least based on msbuild.log - the process doesn't get far enough to run the UCRT version check.

Using Win10 for the moment, under "Program Files (x86)" I do seem to have some older UCRT SDKs (in "Microsoft SDKs/Windows Kits/10/ExtensionSDKs/Microsoft.UniversalCRT.Debug") as 10.0.10150.0, 10.0.10240.0 and 10.0.16299.0.  Whereas only 10.0.16299.0 exists in the "Windows Kits/10/Platforms/UAP" directory.

So I guess msbuild is not strictly picking the "earliest" for UCRT as 10240 isn't quite the earliest.

In comparison, my Windows 7 worker also has the 15063 UCRT SDK so the enforced minimum works there for the UCRT path.  Of course, the ucrtused test in the build process seems to always say 16299.

I ran the requested msbuild command on the Win10 worker as part of PCBuild\build.bat just after the first find_msbuild.bat call.  I'm attaching the log files as msbuild-win10-good.log (commit prior to this change) and msbuild-win10-bad.log (after this change).

So I guess the question is exactly what do we need to enforce in the build process and/or provide at a minimum on a worker?  Should I just install the older 15063 SDK on Win8/10, or should we include some more recent versions in the build time version check?  What is a new user trying to build Python likely to have (assuming they're still using VS2015).

(and the working log)

A quick look at the good build log does indicate that the 8.1 SDK was being used (vars UseWindows81SDK=true and WindowsTargetPlatformVersion=8.1) which is the default behavior for VS2015 if a matching (from python.props) Win10SDK cannot be found.

It seems that the build tools you have installed do not give an error when the selected SDK (WindowsTargetPlatformVersion) isn't found on the system.

You've said that you have the Build Tools for VS2015 installed, a quick look at the download date indicates that they were released prior to VS2015 Update 1 (which includes 10.0.10586.0).

I forgot to mention that the presence of UseWindows81SDK in the build log indicates that the Build Tools are at most at version 1.2 (included with VS2015 Update 1) which should still work (it's what I tested against), but the difference may be the standalone Build Tools might not include an error for missing SDK whereas the VS2015 Tools do.

Hmm, VS2015 started as a full installation (with UI), probably right from its initial release.  The build tools only installation (v141) is for VS2017.

Best I can tell I'm at update 1 - my update version in the registry is 14.0.24720 (plus I have a .1 installer still in the filesystem).  Not sure why I don't have 10586 if that came with update 1.  But I wouldn't be surprised if I'm no further.  The last time I used the VS2015 installer it completely messed up one of my workers and took a bunch of time to recover from.

When switching to VS2017 I think I thought it was going to be preferred for the 3.x series when available, so I probably treated the VS2015 version as less critical.  I guess that didn't end up happening for earlier than 3.7.

Anyway, best next steps to close out this issue...

I assume my installing any matching SDK (say 15063) would allow the workers to match the build script.  Alternatively, including more recent SDKs (16299 is the next in line) in the build script would work with the existing workers.  I'm fine either way.

Well, if VS2015 is installed, the simplest way to have the required SDK(s) is to go to Control Panel -> Uninstall -> Microsoft Visual Studio (Community) 2015

Click Modify.
Expand "Windows and Web Development"
Expand "Universal Windows App Development Tools"
Select at least "Windows 10 SDK (10.0.10586)"

Oh, it's not the installation itself, I'm just wondering if allowing a newer version is ok too?

Of course, it doesn't preclude expanding the build script in the future, so I've installed 15063 to both Win8/10 workers.

I don't currently have access to restart builds through the buildbot web interface any more, but if someone who does wants to rerun the last 3.6 build on each builder we can make sure this resolves the issue on those workers.  

I've done a quick manual build that compiles with the exception of not finding midl.exe for pyshellext, but that seems separate (every SDK has the binary) so I'm assuming it's environmental outside of the regular build environment.

> Oh, it's not the installation itself, I'm just wondering if allowing a newer version is ok too?

The original PR (included in 3.7, 3.x) uses the latest discovered SDK,
but Steve stated that that logic broke the Pipelines build, so I
reworked to PR to just check for the versions already hard-coded.

> Of course, it doesn't preclude expanding the build script in the future, so I've installed 15063 to both Win8/10 workers.

For 3.6, this is the last hurrah.  It goes into security mode after
tonight.  Although, it does seem that we might need to add a check for
the SDK ourselves instead of relying on MSBuild to do it (it may be a
full Visual Studio only thing, not Build Tools).

> I don't currently have access to restart builds through the buildbot web interface any more

If anyone with the power to do so is watching, this is something that
I miss now, too.  It seems that the buildbot owners are not given the
proper permissions for this task.

Ah, got it (and see the pipelines comment by Steve).

Jeremy, I suspect you might actually be able to restart the most recent 3.6 builds on my builders since you were the committer.  It changed in Sep to only allow python-core users and the "owner" of the build.  Though I don't think I learned for sure if owner was the author of the commit or just the actual committer.

I think Zachary was going to look into it time permitting.  We probably need a different GitHub group for buildbot owners or something.

Separately, I think in the context of this issue I think it's fair to say that the commit need not be rolled back.  I have to imagine that anyone trying to build 3.6 using VS 2015 at this point is going to be better represented by Jeremy's worker than mine (in terms of VS point release and default SDK installation), and I expect to be able to resolve any remaining issues worker-side once some 3.6 builds go through.

The 3.6 change broke the compilation 2 Windows buildbots: Jemery Kloth is investigating the regression:
https://github.com/python/cpython/pull/11009#issuecomment-445985774

All the Windows 3.6 windows buildbots are happy! Thanks to all!