classification
Title: GCC detection in setup.py is broken
Type: compile error Stage:
Components: Build Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alex Grund, doko, vstinner
Priority: normal Keywords:

Created on 2019-10-14 12:28 by Alex Grund, last changed 2019-10-21 14:27 by vstinner.

Messages (3)
msg354629 - (view) Author: Alex Grund (Alex Grund) Date: 2019-10-14 12:28
`setup.py` runs `<CC> -E -v - </dev/null 1>/dev/null` to figure out include and library paths from the compiler in the function `add_gcc_paths`.

However sample output from the compiler is:

Es werden eingebaute Spezifikationen verwendet.
COLLECT_GCC=g++
Ziel: x86_64-pc-linux-gnu
Konfiguriert mit: ../configure --enable-languages=c,c++,fortran --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-plugins --enable-gold=default --enable-ld --with-plugin-ld=ld.gold --prefix=/sw/installed/GCCcore/9.1.0 --with-local-prefix=/sw/installed/GCCcore/9.1.0 --enable-bootstrap --with-isl=/dev/shm/easybuild-build/GCCcore/9.1.0/dummy-/gcc-9.1.0/stage2_stuff
Thread-Modell: posix
gcc-Version 9.1.0 (GCC) 
COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /software/haswell/GCCcore/9.1.0/bin/../libexec/gcc/x86_64-pc-linux-gnu/9.1.0/cc1 -E -quiet -v -iprefix /software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/ - -mtune=generic -march=x86-64
nicht vorhandenes Verzeichnis »/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../../x86_64-pc-linux-gnu/include« wird ignoriert
doppeltes Verzeichnis »/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/9.1.0/include« wird ignoriert
doppeltes Verzeichnis »/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/9.1.0/include-fixed« wird ignoriert
nicht vorhandenes Verzeichnis »/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../../x86_64-pc-linux-gnu/include« wird ignoriert
doppeltes Verzeichnis »/sw/installed/GCCcore/9.1.0/include« wird ignoriert
  da es ein Nicht-Systemverzeichnis ist, das ein Systemverzeichnis dupliziert
Suche für »#include "..."« beginnt hier:
Suche für »#include <...>« beginnt hier:
 /sw/installed/binutils/2.32-GCCcore-9.1.0/include
 /sw/installed/zlib/1.2.11-GCCcore-9.1.0/include
 /software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/include
 /software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/include-fixed
 /sw/installed/GCCcore/9.1.0/include
 /usr/include
Ende der Suchliste.
COMPILER_PATH=/software/haswell/GCCcore/9.1.0/bin/../libexec/gcc/x86_64-pc-linux-gnu/9.1.0/:/software/haswell/GCCcore/9.1.0/bin/../libexec/gcc/
LIBRARY_PATH=/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/:/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/:/sw/installed/GCCcore/9.1.0/lib64/../lib64/:/sw/installed/GCCcore/9.1.0/lib/../lib64/:/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/sw/installed/binutils/2.32-GCCcore-9.1.0/lib/:/sw/installed/zlib/1.2.11-GCCcore-9.1.0/lib/:/sw/installed/GCCcore/9.1.0/lib64/:/sw/installed/GCCcore/9.1.0/lib/:/software/haswell/GCCcore/9.1.0/bin/../lib/gcc/x86_64-pc-linux-gnu/9.1.0/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'


So the correct matcher would be "gcc-Version", maybe in addition to "gcc version"

Due to this the setup fails to build e.g. bzip2 modules as the include and lib paths are passed in via CPATH and LIBRARY_PATH only (module system on HPC system)
msg355072 - (view) Author: Alex Grund (Alex Grund) Date: 2019-10-21 14:12
This seems to be a locale issue. So a solution would be to use `LC_ALL=C` before invoking the compiler (or the cross-platform equivalent)
msg355074 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-21 14:27
What are you trying to do? Compile Python? Cross-compile Python?

> `setup.py` runs `<CC> -E -v - </dev/null 1>/dev/null` to figure out include and library paths from the compiler in the function `add_gcc_paths`.

Are you talking about the add_cross_compiling_paths() function which is only called when _PYTHON_HOST_PLATFORM environment variable is set?

So GCC says "gcc-Version 9.1.0 (GCC) " with your locale, but does it write "gcc version 9.1.0 ..." with LC_ALL=C?
History
Date User Action Args
2019-10-21 14:27:28vstinnersetnosy: + vstinner
messages: + msg355074
2019-10-21 14:12:26Alex Grundsetmessages: + msg355072
2019-10-14 12:30:28ned.deilysetnosy: + doko
2019-10-14 12:28:34Alex Grundcreate