Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found #79480

Closed
neyuru mannequin opened this issue Nov 23, 2018 · 6 comments
Closed

LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found #79480

neyuru mannequin opened this issue Nov 23, 2018 · 6 comments
Assignees
@zooba
Labels
3.7 (EOL) end of life 3.8 only security fixes build The build process and cross-build OS-windows stdlib Python modules in the Lib dir tests Tests in the Lib/test dir

Comments

@neyuru
Copy link
Mannequin

neyuru mannequin commented Nov 23, 2018

BPO 35299
Nosy @pfmoore, @tjguk, @merwok, @zware, @zooba, @dstufft, @miss-islington, @neyuru
PRs
  • bpo-35299: Fixes sysconfig and distutils during PGO profiling #11744
  • bpo-35299: Fixes sysconfig and distutils during PGO profiling #11744
  • bpo-35299: Fixes sysconfig and distutils during PGO profiling #11744
  • [3.7] bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744) #11758
  • [3.7] bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744) #11758
  • [3.7] bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744) #11758
    		•
    Files
  • LIBenRepogiT.txt: Verbose of the compilation with the steps shown
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/zooba'
closed_at = <Date 2019-02-05.01:55:19.153>
created_at = <Date 2018-11-23.06:31:25.303>
labels = ['3.7', '3.8', 'library', 'build', 'tests', 'OS-windows']
title = "LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found"
updated_at = <Date 2019-02-05.01:55:17.147>
user = 'https://github.com/neyuru'

    bugs.python.org fields:

    activity = <Date 2019-02-05.01:55:17.147>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2019-02-05.01:55:19.153>
closer = 'steve.dower'
components = ['Distutils', 'Tests', 'Windows']
creation = <Date 2018-11-23.06:31:25.303>
creator = 'neyuru'
dependencies = []
files = ['47941']
hgrepos = []
issue_num = 35299
keywords = ['patch', 'patch', 'patch']
message_count = 6.0
messages = ['330299', '330347', '334040', '334780', '334853', '334855']
nosy_count = 8.0
nosy_names = ['paul.moore', 'tim.golden', 'eric.araujo', 'zach.ware', 'steve.dower', 'dstufft', 'miss-islington', 'neyuru']
pr_nums = ['11744', '11744', '11744', '11758', '11758', '11758']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue35299'
versions = ['Python 3.7', 'Python 3.8']
    @neyuru
    Copy link
    Mannequin Author

    neyuru mannequin commented Nov 23, 2018

    Build of Python 3.6.7 fails with error
    "LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found"
    I am using Windows 10 64bit.

    Steps to reproduce this error:

    1. Install Visual Studio 2017 (With the python development and native dev tools) (I have also installed NVIDIA CUDA Toolkit but I don't think this matters)
    2. Download source code from Python to folder E:\RepoGit\3.6 (this is the download folder)
    3. Open a command prompt, navigate to "E:\RepoGit" and run "PCBuild\build.bat -p x64"
    4. Close the command prompt
    5. Copy file pyconfig.h file from E:\RepoGit\3.6\PC and paste it to E:\RepoGit\3.6\include. Also, create an environment variable called "LIB" that contains the folder "E:\RepoGiT\3.6\PCbuild\amd64" (this folder was created at step 3)
    6. Open a command prompt and run "Tools\msi\buildrelease.bat -x64"

    NOTE1: if step 5 is skipped, the overall build does not fail, but the PGInstrument x64 Interpreter fails in distutils with error "E:\RepoGiT\3.6\include\Python.h(8): fatal error C1083: Cannot open include file: 'pyconfig.h': No such file or directory", as shown in message https://bugs.python.org/msg329247 of issue https://bugs.python.org/issue35157

    NOTE2: if pyconfig.h in step 5 is copied but the environment variable is not created, the PGInstrument x64 Interpreter fails in distutils with error "LINK : fatal error LNK1104: cannot open file 'python36.lib'", and the overall build fails with the same error shown in the title of this Issue.

    NOTE3: If all the steps are run, the PGInstrument x64 Interpreter succeeds in its tests but the overall build fails with the error shown in the title of this issue. All the verbose of this compilation is included in the file attached.

    @neyuru neyuru mannequin added build The build process and cross-build OS-windows labels Nov 23, 2018
    @zooba
    Copy link
    Member

    zooba commented Nov 23, 2018

    I'm going to declare that this is a bug in distutils not correctly setting INCLUDE and LIB values when building in a source directory. When this is fixed, there will be no need to copy pyconfig.h anywhere, and the installer build will be fine.

    (And it should be noted that we don't rely on 100% pass rate of the test suite when training for PGO - the point is to exercise as many code paths as possible. The test suite has shown to be as good as any other generic workload, and is easily available in our repo.)

    @zooba zooba added stdlib Python modules in the Lib dir tests Tests in the Lib/test dir 3.7 (EOL) end of life 3.8 only security fixes and removed build The build process and cross-build labels Nov 23, 2018
    @neyuru
    Copy link
    Mannequin Author

    neyuru mannequin commented Jan 19, 2019

    I want to document a workaround if anyone benefits from it, this would be step 7:

    1. This error goes away if you remove the pyconfig.h from the include directory before the tests end (and of course, after the test distutils).

    This is only a workaround and requires the user placing attention to the build process (maybe for the first 5-10 minutes). Having the option "-q" removed from the buildrelease.bat file (as discussed in issue https://bugs.python.org/issue35739 ) helps with determining when to safely remove the file so that the distutils test pass.

    @zooba
    Copy link
    Member

    zooba commented Feb 2, 2019

    My PR fixes both issues properly, but adds a minor "feature" to distutils by making distutils.sysconfig.get_python_inc() potentially return a set of include paths (with os.path.pathsep separators).

    This is fully internal, so right now we know it'll only occur when building in the source tree on Windows, so I'm not concerned about it. It'd also work on any platform through an environment variable, which it is, but it looks like most people simply attach "-I" at the start and pass it to gcc (which again, since they're not on Windows, they're not going to hit the one case we know about).

    I'd rather do it properly than have a one-time hack just for this build scenario, and I certainly don't want to be copying files within the source tree. So I'm intending to go ahead with this design, but wanted to allow any interested parties to push back if they know of some reason that I don't.

    @zooba
    Copy link
    Member

    zooba commented Feb 5, 2019

    New changeset 85e102a by Steve Dower in branch 'master':
    bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744)
    85e102a

    @zooba zooba self-assigned this Feb 5, 2019
    @miss-islington
    Copy link
    Contributor

    New changeset 4f6854a by Miss Islington (bot) in branch '3.7':
    bpo-35299: Fixed sysconfig and distutils during PGO profiling (GH-11744)
    4f6854a

    @zooba zooba closed this as completed Feb 5, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @zooba zooba

    Labels
    3.7 (EOL) end of life 3.8 only security fixes build The build process and cross-build OS-windows stdlib Python modules in the Lib dir tests Tests in the Lib/test dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @zooba @miss-islington