classification
Title: _msvccompiler.py doesn't properly support manifests
Type: behavior Stage: resolved
Components: Distutils, Windows Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: cgohlke, dstufft, eric.araujo, gladman, paul.moore, python-dev, r.david.murray, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2015-08-05 16:25 by gladman, last changed 2015-08-14 23:37 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
24798_1.patch steve.dower, 2015-08-05 18:47 review
Messages (9)
msg248050 - (view) Author: (gladman) Date: 2015-08-05 16:25
I have been using _msvcompiler.py from Python 3.5 to build some executables but I have been unable to get it to generate and embed a  manifest.  

When I looked into this I found that the subroutine that sets up the parameters for generating a manifest ('manifest_get_embed_info'  at around line 471) has the line:

        ld_args.append('/MANIFESTFILE:' + temp_manifest)

to set the manifest's name but doesn't actually ask for a manifest to be generated.  Here is what is said about /MANIFESTFILE on MSDN:

"/MANIFESTFILE will have no effect if you do not also link with /MANIFEST."

After adding:

        ld_args.append('/MANIFEST')

before the above line, I then succeed in obtaining the manifest.
msg248052 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-08-05 17:28
It looks like manifests aren't generated with CRT version info anymore either, so I'll take this opportunity to clean up the build process some more too.
msg248057 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-08-05 18:47
I've simplified the manifest handling:

* embed by default, since we no longer need to filter CRT SxS info
* omit UAC info from DLLs
* use full PATH when running tools
* some other tidying

Not necessarily going to wait for lots of reviews, as I know very few people are interested in distutils, but feel free to chime in if you like. I'll give it a day or so.
msg248230 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-08-07 21:39
New changeset 8e966eba2b5e by Steve Dower in branch '3.5':
Issue #24798: _msvccompiler.py doesn't properly support manifests
https://hg.python.org/cpython/rev/8e966eba2b5e

New changeset f61a083b843f by Steve Dower in branch 'default':
Issue #24798: _msvccompiler.py doesn't properly support manifests
https://hg.python.org/cpython/rev/f61a083b843f
msg248231 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-08-07 21:43
Made a couple more changes (including a fix for vcruntime140.dll embedding) after testing numpy's build, but that worked fine with the final fixes.
msg248412 - (view) Author: Christoph Gohlke (cgohlke) Date: 2015-08-11 16:23
This change broke all my builds that link statically against 3rd party libraries built with the `/MD` flag. `/MD` was used at least since Python 2.3 and is the default for static libraries in Visual Studio 2015. Some of the broken builds: lxml, pillow, matplotlib, pygame, pycuda, pymssql, netcdf4, GDAL, psycopg2, pycurl, gmpy, and pyopenssl. All of these packages built OK with Python 3.5.0b4.

The build errors are of this kind: 
`error LNK2001: unresolved external symbol __imp_memchr`
`error LNK2001: unresolved external symbol __imp_strstr`

The linker throws the following warning:
`LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library`
msg248413 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-11 16:36
Please open a new issue, referencing this one.  Priority should be set to release blocker.  (I forget if regular users can do that; if you can't I will.)
msg248621 - (view) Author: Christoph Gohlke (cgohlke) Date: 2015-08-14 22:22
It seems the switch to '/MT' was consciously intended as Python 3.5 itself is now compiled with '/MT'. 
For now I have patched _msvccompiler.py to use '/MD' and continue to link libraries built with '/MD'.
msg248622 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-08-14 23:37
Those libraries will depend on vcruntime140.dll which is not installed with CPython right now. In order for packages built with future compilers to work we need to statically link that dependency but not ucrt, which there are linker options for.

I'm thinking hard to come up with better approaches, but most of them require predicting the future. Create a new issue if you like or I'll make one when I have something to propose.
History
Date User Action Args
2015-08-14 23:37:57steve.dowersetmessages: + msg248622
2015-08-14 22:22:38cgohlkesetmessages: + msg248621
2015-08-11 16:36:50r.david.murraysetnosy: + r.david.murray
messages: + msg248413
2015-08-11 16:23:02cgohlkesetnosy: + cgohlke
messages: + msg248412
2015-08-07 21:43:20steve.dowersetstatus: open -> closed
resolution: fixed
messages: + msg248231

stage: resolved
2015-08-07 21:39:41python-devsetnosy: + python-dev
messages: + msg248230
2015-08-05 18:47:23steve.dowersetfiles: + 24798_1.patch
keywords: + patch
messages: + msg248057
2015-08-05 17:28:07steve.dowersetmessages: + msg248052
2015-08-05 16:47:55zach.waresetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
title: Issue in the MSVC compiler class in distutils on Python 3.5 -> _msvccompiler.py doesn't properly support manifests
assignee: steve.dower
versions: + Python 3.6
components: + Windows
2015-08-05 16:25:34gladmancreate