classification
Title: LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found
Type: compile error Stage: resolved
Components: Distutils, Tests, Windows Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: dstufft, eric.araujo, miss-islington, neyuru, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch, patch, patch

Created on 2018-11-23 06:31 by neyuru, last changed 2019-02-05 01:55 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
LIBenRepogiT.txt neyuru, 2018-11-23 06:31 Verbose of the compilation with the steps shown
Pull Requests
URL Status Linked Edit
PR 11744 merged steve.dower, 2019-02-02 22:48
PR 11744 merged steve.dower, 2019-02-02 22:48
PR 11744 merged steve.dower, 2019-02-02 22:48
PR 11758 merged miss-islington, 2019-02-05 01:15
PR 11758 merged miss-islington, 2019-02-05 01:15
PR 11758 merged miss-islington, 2019-02-05 01:15
Messages (6)
msg330299 - (view) Author: Jorge Ramos (neyuru) * Date: 2018-11-23 06:31
Build of Python 3.6.7 fails with error 
"LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found" 
I am using Windows 10 64bit.

Steps to reproduce this error:
1) Install Visual Studio 2017 (With the python development and native dev tools) (I have also installed NVIDIA CUDA Toolkit but I don't think this matters)
2) Download source code from Python to folder E:\RepoGit\3.6 (this is the download folder)
3) Open a command prompt, navigate to "E:\RepoGit" and run "PCBuild\build.bat -p x64"
4) Close the command prompt
5) Copy file pyconfig.h file from E:\RepoGit\3.6\PC and paste it to E:\RepoGit\3.6\include. Also, create an environment variable called "LIB" that contains the folder "E:\RepoGiT\3.6\PCbuild\amd64" (this folder was created at step 3) 
6) Open a command prompt and run "Tools\msi\buildrelease.bat -x64"

NOTE1: if step 5 is skipped, the overall build does not fail, but the PGInstrument x64 Interpreter fails in distutils with error "E:\RepoGiT\3.6\include\Python.h(8): fatal error C1083: Cannot open include file: 'pyconfig.h': No such file or directory", as shown in message https://bugs.python.org/msg329247 of issue https://bugs.python.org/issue35157

NOTE2: if pyconfig.h in step 5 is copied but the environment variable is not created, the PGInstrument x64 Interpreter fails in distutils with error "LINK : fatal error LNK1104: cannot open file 'python36.lib'", and the overall build fails with the same error shown in the title of this Issue.

NOTE3: If all the steps are run, the PGInstrument x64 Interpreter succeeds in its tests but the overall build fails with the error shown in the title of this issue. All the verbose of this compilation is included in the file attached.
msg330347 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-11-23 17:55
I'm going to declare that this is a bug in distutils not correctly setting INCLUDE and LIB values when building in a source directory. When this is fixed, there will be no need to copy pyconfig.h anywhere, and the installer build will be fine.

(And it should be noted that we don't rely on 100% pass rate of the test suite when training for PGO - the point is to exercise as many code paths as possible. The test suite has shown to be as good as any other generic workload, and is easily available in our repo.)
msg334040 - (view) Author: Jorge Ramos (neyuru) * Date: 2019-01-19 06:34
I want to document a workaround if anyone benefits from it, this would be step 7:

7) This error goes away if you remove the pyconfig.h from the include directory before the tests end (and of course, after the test distutils).

This is only a workaround and requires the user placing attention to the build process (maybe for the first 5-10 minutes). Having the option "-q" removed from the buildrelease.bat file (as discussed in issue https://bugs.python.org/issue35739 ) helps with determining when to safely remove the file so that the distutils test pass.
msg334780 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-02-02 23:55
My PR fixes both issues properly, but adds a minor "feature" to distutils by making distutils.sysconfig.get_python_inc() potentially return a set of include paths (with os.path.pathsep separators).

This is fully internal, so right now we know it'll only occur when building in the source tree on Windows, so I'm not concerned about it. It'd also work on any platform through an environment variable, which it is, but it looks like most people simply attach "-I" at the start and pass it to gcc (which again, since they're not on Windows, they're not going to hit the one case we know about).

I'd rather do it properly than have a one-time hack just for this build scenario, and I certainly don't want to be copying files within the source tree. So I'm intending to go ahead with this design, but wanted to allow any interested parties to push back if they know of some reason that I don't.
msg334853 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-02-05 01:15
New changeset 85e102a2b090dd693d0801ae2edb9660cfa0f281 by Steve Dower in branch 'master':
bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744)
https://github.com/python/cpython/commit/85e102a2b090dd693d0801ae2edb9660cfa0f281
msg334855 - (view) Author: miss-islington (miss-islington) Date: 2019-02-05 01:55
New changeset 4f6854a690e95ab5159687794652c639a020d6fd by Miss Islington (bot) in branch '3.7':
bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744)
https://github.com/python/cpython/commit/4f6854a690e95ab5159687794652c639a020d6fd
History
Date User Action Args
2019-02-05 01:55:19steve.dowersetkeywords: patch, patch, patch
status: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-02-05 01:55:03miss-islingtonsetnosy: + miss-islington
messages: + msg334855
2019-02-05 01:15:46steve.dowersetkeywords: patch, patch, patch
assignee: steve.dower
2019-02-05 01:15:36miss-islingtonsetpull_requests: + pull_request11705
2019-02-05 01:15:34miss-islingtonsetpull_requests: + pull_request11706
2019-02-05 01:15:32miss-islingtonsetpull_requests: + pull_request11704
2019-02-05 01:15:17steve.dowersetmessages: + msg334853
2019-02-02 23:55:09steve.dowersetkeywords: patch, patch, patch

messages: + msg334780
versions: - Python 3.6
2019-02-02 22:48:59steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request11666
2019-02-02 22:48:50steve.dowersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request11665
2019-02-02 22:48:41steve.dowersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request11664
2019-01-19 06:34:11neyurusetmessages: + msg334040
2018-11-23 17:55:59steve.dowersetversions: + Python 3.7, Python 3.8
nosy: + eric.araujo, dstufft

messages: + msg330347

components: + Distutils, Tests, - Build
2018-11-23 06:31:25neyurucreate