classification
Title: distutils sometimes assumes wrong C compiler
Type: Stage:
Components: Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: moxian
Priority: normal Keywords:

Created on 2017-07-12 21:43 by moxian, last changed 2017-07-12 21:43 by moxian.

Messages (1)
msg298248 - (view) Author: moxian (moxian) Date: 2017-07-12 21:43
https://github.com/python/cpython/blob/2.7/Lib/distutils/unixccompiler.py#L229
In unixcompiler.py runtime_library_dir_option() function, we get the name of active C compiler the following way: `compiler = os.path.basename(sysconfig.get_config_var("CC"))`

This works fine usually, but this gets the compiler that was used while building python itself, which might be different from the one used to build python extension right now.
I.e. if we've built the cpython with clang, but want to build some other extension with gcc, we would run `CC=gcc python setup.py build`, passing gcc as the environment variable. The environment variable would be picked up, and override the ones used for compilation of cpython here: https://github.com/python/cpython/blob/2.7/Lib/distutils/sysconfig.py#L179 , so that the compiler binary will be set to "gcc". But the options passed to the compiler will still assume it to be clang, and, e.g. runtime_library_dir_option() mentioned above would fall back to returning catch-all `"-R" + dir` option, since `compiler` is not overriden by environment vars.
This breaks compilation, since gcc does not understand how to treat "-R" flag. Example from trying to build bsddb3:

gcc-5.4.0 -shared -Wl,-O1 -Wl,--as-needed -O2 -pipe -march=native -ggdb -fno-strict-aliasing /var/tmp/portage/dev-python/bsddb3-6.1.1/work/bsddb3-6.1.1-python2_7/build/temp.linux-x86_64-2.7/Modules/_bsddb.o -L/usr/lib64 -L/usr/lib64 -R/usr/lib64 -ldb-5.3 -lpython2.7 -o /var/tmp/portage/dev-python/bsddb3-6.1.1/work/bsddb3-6.1.1-python2_7/build/lib/bsddb3/_pybsddb.so
gcc-5.4.0: error: unrecognized command line option ‘-R’

I've personally only encountered it with runtime_library_dir_option(), but I presume there are other occurences that get overriden by environment variables only in sysconfig.py, but not in other places.
The problem is present in 2.7 and *I think* in 3.6 as well.

I think the correct fix would be for get_config_vars() to always respect environment variables, but I don't know enough about the code to feel confident about that.
History
Date User Action Args
2017-07-12 21:43:23moxiancreate