➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: distutils sometimes assumes wrong C compiler
Type: Stage: resolved
Components: Distutils Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: dstufft, eric.araujo, moxian, steve.dower
Priority: normal Keywords:

Created on 2017-07-12 21:43 by moxian, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (2)
msg298248 - (view) Author: moxian (moxian) Date: 2017-07-12 21:43
https://github.com/python/cpython/blob/2.7/Lib/distutils/unixccompiler.py#L229
In unixcompiler.py runtime_library_dir_option() function, we get the name of active C compiler the following way: `compiler = os.path.basename(sysconfig.get_config_var("CC"))`

This works fine usually, but this gets the compiler that was used while building python itself, which might be different from the one used to build python extension right now.
I.e. if we've built the cpython with clang, but want to build some other extension with gcc, we would run `CC=gcc python setup.py build`, passing gcc as the environment variable. The environment variable would be picked up, and override the ones used for compilation of cpython here: https://github.com/python/cpython/blob/2.7/Lib/distutils/sysconfig.py#L179 , so that the compiler binary will be set to "gcc". But the options passed to the compiler will still assume it to be clang, and, e.g. runtime_library_dir_option() mentioned above would fall back to returning catch-all `"-R" + dir` option, since `compiler` is not overriden by environment vars.
This breaks compilation, since gcc does not understand how to treat "-R" flag. Example from trying to build bsddb3:

gcc-5.4.0 -shared -Wl,-O1 -Wl,--as-needed -O2 -pipe -march=native -ggdb -fno-strict-aliasing /var/tmp/portage/dev-python/bsddb3-6.1.1/work/bsddb3-6.1.1-python2_7/build/temp.linux-x86_64-2.7/Modules/_bsddb.o -L/usr/lib64 -L/usr/lib64 -R/usr/lib64 -ldb-5.3 -lpython2.7 -o /var/tmp/portage/dev-python/bsddb3-6.1.1/work/bsddb3-6.1.1-python2_7/build/lib/bsddb3/_pybsddb.so
gcc-5.4.0: error: unrecognized command line option ‘-R’

I've personally only encountered it with runtime_library_dir_option(), but I presume there are other occurences that get overriden by environment variables only in sysconfig.py, but not in other places.
The problem is present in 2.7 and *I think* in 3.6 as well.

I think the correct fix would be for get_config_vars() to always respect environment variables, but I don't know enough about the code to feel confident about that.
msg386540 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-05 21:29
Existing distutils bugs are closed under PEP 632. New issues will be considered if they are release blockers, but (by definition) no existing issue meets this bar.
History
Date User Action Args
2022-04-11 14:58:49adminsetgithub: 75098
2021-02-05 21:29:13steve.dowersetstatus: open -> closed
resolution: out of date
messages: + msg386540

stage: resolved
2021-02-05 18:38:53iritkatrielsetnosy: + eric.araujo, dstufft, steve.dower
components: + Distutils
2017-07-12 21:43:23moxiancreate