Title: Some python extensions can't be compiled with clang 3.4
Type: compile error Stage: patch review
Components: Distutils Versions: Python 3.6, Python 3.5, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Antoine.Brodin.FreeBSD, Arfrever, djc, doko, dstufft, eric.araujo, ezio.melotti, haypo, koobs, loewis, paul.moore, skrah, steve.dower, thomas-petazzoni, tim.golden, zach.ware
Priority: high Keywords: easy, needs review, patch

Created on 2014-02-25 13:33 by Antoine.Brodin.FreeBSD, last changed 2016-04-12 18:52 by zach.ware.

File name Uploaded Description Edit
python-issue20767.diff koobs, 2014-02-28 09:04 review
Messages (13)
msg212176 - (view) Author: Antoine Brodin.FreeBSD (Antoine.Brodin.FreeBSD) Date: 2014-02-25 13:33

On FreeBSD -current,  clang 3.4 is now the default compiler.
Clang 3.4 rejects "-R/path/to/lib" flag (previously in version 3.3 it just ignored it).

This leads to some errors with some python extensions:

cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cache.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/connection.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cursor.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/microprotocols.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/module.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/prepare_protocol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/row.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/statement.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/util.o -L/usr/local/lib -R/usr/local/lib -lsqlite3 -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/
cc: error: unknown argument: '-R/usr/local/lib'
error: command 'cc' failed with exit status 1

cc -shared -O2 -pipe -DLDAP_DEPRECATED -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/LDAPObject.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapcontrol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/common.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/constants.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/errors.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/functions.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/schema.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapmodule.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/message.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/version.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/options.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/berval.o -L/usr/local/lib -L/usr/lib -R/usr/local/lib -R/usr/lib -lldap_r -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/
cc: error: unknown argument: '-R/usr/local/lib'
cc: error: unknown argument: '-R/usr/lib'
error: command 'cc' failed with exit status 1

cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/_bsddb.o -L/usr/local/lib -R/usr/local/lib -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/bsddb3/ -ldb-4.3
cc: error: unknown argument: '-R/usr/local/lib'
error: command 'cc' failed with exit status 1

I found the patch below to help, it is for function runtime_library_dir_option in Lib/distutils/
-Wl,-rpath works for both gcc and clang on freebsd

--- ./Lib/distutils/
+++ ./Lib/distutils/
@@ -228,6 +228,8 @@
         if sys.platform[:6] == "darwin":
             # MacOSX's linker doesn't understand the -R flag at all
             return "-L" + dir
+        elif sys.platform[:7] == "freebsd":
+            return "-Wl,-rpath=" + dir
         elif sys.platform[:5] == "hp-ux":
             if self._is_gcc(compiler):
                 return ["-Wl,+s", "-L" + dir]
msg212177 - (view) Author: koobs (koobs) Date: 2014-02-25 13:40
Details on how clang 3.4 changes behaviour for compiler flags:
msg212185 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-02-25 15:48
this looks safe from my point of view.

However the real problem is that you unconditionally add a runtime path for a standard system path.  I think the better way to fix this is not to pass the -L and -R arguments at all if the library is found in a system path.
msg212188 - (view) Author: Antoine Brodin.FreeBSD (Antoine.Brodin.FreeBSD) Date: 2014-02-25 16:45
For the python-ldap extension, this seems to be a buglet in its setup.cfg, it lists /usr/lib in library_dirs and /usr/include in library_dirs

For the others, /usr/local/lib is not in the default library search path (only /lib and /usr/lib) so at least -L has to be specified.
msg212193 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-02-25 18:26
doko: how do you know the addition of the -R option is unconditional? and whom do you refer to by "you" who is adding the option?

In any case, the patch is independent of whether the option is added unconditionally, and I agree that the patch looks safe. The question is whether it should be extended to the other *BSDs as well.
msg212415 - (view) Author: koobs (koobs) Date: 2014-02-28 09:04
Attaching patch against default
msg212419 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-02-28 10:32
>  The question is whether it should be extended to the other *BSDs as well.

The change is not needed on Linux (if Clang 3.4 is used to compile extensions too)?
msg212718 - (view) Author: koobs (koobs) Date: 2014-03-04 13:23
It would be great if this could make it into 3.4, extended to include other OS's if necessary at a later date.
msg212719 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-03-04 13:38 suggests that this problem occurs also on Linux when using Clang.
msg215310 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2014-04-01 13:12
Can we have some more feedback on this?
msg215312 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-01 13:16
See also issue #21122.
msg215315 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2014-04-01 13:29
That does look to be a different issue, though.
msg257021 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2015-12-26 10:27
Over at the llvm bug tracker this is marked as a release blocker:
Date User Action Args
2016-04-12 18:52:57zach.waresethgrepos: - hgrepo339
2016-04-12 16:41:12berker.peksagsetstage: patch review
components: - Build, Extension Modules, Interpreter Core, Tkinter, Unicode, Windows, XML
versions: - Python 3.2, Python 3.3, Python 3.4
2016-04-12 15:46:51supriyantomaftuhsethgrepos: + hgrepo339

nosy: + ezio.melotti, paul.moore, tim.golden, eric.araujo, zach.ware, dstufft, steve.dower
components: + Build, Extension Modules, Interpreter Core, Tkinter, Unicode, Windows, XML
versions: + Python 3.2
2015-12-26 10:27:56skrahsetnosy: + skrah
messages: + msg257021
2015-12-26 02:43:19koobssetkeywords: + easy, needs review
versions: + Python 3.6
2014-04-01 13:29:58djcsetmessages: + msg215315
2014-04-01 13:16:22hayposetmessages: + msg215312
2014-04-01 13:12:52djcsetmessages: + msg215310
2014-04-01 13:12:27djcsetnosy: + djc
2014-03-04 13:38:57Arfreversetmessages: + msg212719
2014-03-04 13:23:03koobssetmessages: + msg212718
2014-02-28 10:32:18hayposetnosy: + haypo
messages: + msg212419
2014-02-28 09:04:49koobssetfiles: + python-issue20767.diff
type: compile error
messages: + msg212415

keywords: + patch
2014-02-26 04:50:02Arfreversetnosy: + Arfrever
2014-02-25 18:26:18loewissetmessages: + msg212193
2014-02-25 16:45:44Antoine.Brodin.FreeBSDsetmessages: + msg212188
2014-02-25 15:48:31dokosetmessages: + msg212185
2014-02-25 15:44:47rhettingersetpriority: normal -> high
2014-02-25 13:40:55koobssetmessages: + msg212177
2014-02-25 13:38:36koobssetversions: + Python 3.3, Python 3.4, Python 3.5
2014-02-25 13:34:27hayposetnosy: + loewis, doko, thomas-petazzoni
2014-02-25 13:33:18Antoine.Brodin.FreeBSDcreate