classification
Title: test_distutils: test_get_exe_bytes() failure on AppVeyor
Type: Stage: resolved
Components: Tests, Windows Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: high Keywords: patch

Created on 2017-12-13 15:35 by vstinner, last changed 2017-12-14 11:06 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4851 merged vstinner, 2017-12-13 21:39
PR 4861 merged python-dev, 2017-12-14 10:40
Messages (11)
msg308203 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-13 15:35
Example: https://ci.appveyor.com/project/python/cpython/build/3.7.0a0.9414

======================================================================
ERROR: test_get_exe_bytes (distutils.tests.test_bdist_wininst.BuildWinInstTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\distutils\tests\test_bdist_wininst.py", line 24, in test_get_exe_bytes
    exe_file = cmd.get_exe_bytes()
  File "C:\projects\cpython\lib\distutils\command\bdist_wininst.py", line 361, in get_exe_bytes
    f = open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\projects\\cpython\\lib\\distutils\\command\\wininst-14.12.exe'
----------------------------------------------------------------------
msg308207 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-12-13 16:24
I thought we'd special cased the v14 toolset already.

This should be an update to the Python code in distutils that selects the filename based on compiler version.
msg308210 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-13 16:30
Extract of pythoninfo of the failed build:

os.environ[VS110COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\
os.environ[VS120COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\
os.environ[VS140COMNTOOLS]: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\

The build starts with:

Using "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe"  (found in the PATH)
msg308211 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-13 16:30
Oh, and the PATH:

os.environ[PATH]: C:\Perl\site\bin;C:\Perl\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\7-Zip;C:\Tools\GitVersion;C:\Tools\NuGet;C:\Program Files\Microsoft\Web Platform Installer\;C:\Tools\PsTools;C:\Program Files\Git LFS;C:\Program Files\Mercurial\;C:\Program Files (x86)\Subversion\bin;C:\Tools\WebDriver;C:\Tools\Coverity\bin;C:\Tools\MSpec;C:\Tools\NUnit\bin;C:\Tools\NUnit3;C:\Tools\xUnit;C:\Program Files\nodejs;C:\Program Files (x86)\iojs;C:\Program Files\iojs;C:\Users\appveyor\AppData\Roaming\npm;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Ruby193\bin;C:\go\bin;C:\Program Files\Java\jdk1.8.0\bin;C:\Program Files (x86)\Apache\Maven\bin;C:\Python27;C:\Python27\Scripts;C:\Program Files\erl8.3\bin;C:\Program Files (x86)\CMake\bin;C:\Tools\curl\bin;C:\Program Files\LLVM\bin;C:\Users\appveyor\.dnx\bin;C:\Program Files\Microsoft DNX\Dnvm\;C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin;C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\Microsoft\SQLDB\DAC\130;C:\Program Files\Amazon\AWSCLI\;C:\Program Files\dotnet\;C:\Tools\vcpkg;C:\Program Files (x86)\dotnet\;C:\Users\appveyor\AppData\Local\Microsoft\WindowsApps;C:\Users\appveyor\AppData\Local\Yarn\bin;C:\Users\appveyor\AppData\Roaming\npm;C:\Program Files\Docker;C:\Program Files\Git\cmd;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\usr\bin;C:\Program Files (x86)\Yarn\bin;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\nodejs\;C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\TestPlatform;C:\Users\appveyor\AppData\Local\Microsoft\WindowsApps;C:\Users\appveyor\AppData\Local\Yarn\bin;C:\Users\appveyor\AppData\Roaming\npm;C:\Program Files\AppVeyor\BuildAgent\
msg308215 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-12-13 16:42
This is the bit that needs fixing https://github.com/python/cpython/blob/master/Lib/distutils/command/bdist_wininst.py#L340

                bv = '.'.join(CRT_ASSEMBLY_VERSION.split('.', 2)[:2])
                if bv == '14.11':
                    # v141 and v140 are binary compatible,
                    # so keep using the 14.0 stub.
                    bv = '14.0'

AppVeyor has clearly gotten hold of 14.12, which should also be binary compatible (version numbers here are not strictly SemVer, but the 14 indicates binary compatibility).
msg308218 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-13 16:48
@Steve: do you want to work on a fix?
msg308233 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-12-13 19:16
Happy for someone else to do it. I won't have time this week - too much going on at work.
msg308254 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-13 21:44
> Happy for someone else to do it. I won't have time this week - too much going on at work.

Sure. I wrote a PR to fix the exact bug.
msg308284 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-14 10:39
New changeset 9e7c136ad8bc8e8eec50c2a8ae5ff02752f695a2 by Victor Stinner in branch 'master':
bpo-32302: Fix distutils bdist_wininst for CRT v142 (#4851)
https://github.com/python/cpython/commit/9e7c136ad8bc8e8eec50c2a8ae5ff02752f695a2
msg308286 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-14 11:00
New changeset 645fa19f15cddeec8d882a9e1420173e7516f0e4 by Victor Stinner (Miss Islington (bot)) in branch '3.6':
bpo-32302: Fix distutils bdist_wininst for CRT v142 (GH-4851) (#4861)
https://github.com/python/cpython/commit/645fa19f15cddeec8d882a9e1420173e7516f0e4
msg308289 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-14 11:06
It seems like AppVeyor is fixed, thanks Steve for the help!

Later, it would be nice to write a more generic fix than hardcoding two minor versions.
History
Date User Action Args
2017-12-14 11:06:42vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg308289

stage: patch review -> resolved
2017-12-14 11:00:52vstinnersetmessages: + msg308286
2017-12-14 10:40:00python-devsetpull_requests: + pull_request4751
2017-12-14 10:39:37vstinnersetmessages: + msg308284
2017-12-13 21:44:18vstinnersetmessages: + msg308254
2017-12-13 21:39:34vstinnersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request4740
2017-12-13 19:16:19steve.dowersetmessages: + msg308233
2017-12-13 16:48:30vstinnersetmessages: + msg308218
2017-12-13 16:42:41steve.dowersetmessages: + msg308215
2017-12-13 16:30:34vstinnersetmessages: + msg308211
2017-12-13 16:30:00vstinnersetmessages: + msg308210
2017-12-13 16:24:08steve.dowersetmessages: + msg308207
2017-12-13 16:00:34zach.waresetpriority: normal -> high
nosy: + paul.moore, tim.golden, zach.ware, steve.dower

components: + Windows
stage: needs patch
2017-12-13 15:35:38vstinnercreate