classification
Title: Some python extensions can't be compiled with clang 3.4
Type: compile error Stage:
Components: Distutils Versions: Python 3.5, Python 3.4, Python 3.3, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Antoine.Brodin.FreeBSD, Arfrever, djc, doko, haypo, koobs, loewis, thomas-petazzoni
Priority: high Keywords: patch

Created on 2014-02-25 13:33 by Antoine.Brodin.FreeBSD, last changed 2014-04-01 13:29 by djc.

Files
File name Uploaded Description Edit
python-issue20767.diff koobs, 2014-02-28 09:04 review
Messages (12)
msg212176 - (view) Author: Antoine Brodin.FreeBSD (Antoine.Brodin.FreeBSD) Date: 2014-02-25 13:33
Hi,

On FreeBSD -current,  clang 3.4 is now the default compiler.
Clang 3.4 rejects "-R/path/to/lib" flag (previously in version 3.3 it just ignored it).

This leads to some errors with some python extensions:

cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cache.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/connection.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cursor.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/microprotocols.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/module.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/prepare_protocol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/row.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/statement.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/util.o -L/usr/local/lib -R/usr/local/lib -lsqlite3 -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/_sqlite3.so
cc: error: unknown argument: '-R/usr/local/lib'
error: command 'cc' failed with exit status 1


cc -shared -O2 -pipe -DLDAP_DEPRECATED -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/LDAPObject.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapcontrol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/common.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/constants.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/errors.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/functions.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/schema.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapmodule.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/message.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/version.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/options.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/berval.o -L/usr/local/lib -L/usr/lib -R/usr/local/lib -R/usr/lib -lldap_r -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/_ldap.so
cc: error: unknown argument: '-R/usr/local/lib'
cc: error: unknown argument: '-R/usr/lib'
error: command 'cc' failed with exit status 1


cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/_bsddb.o -L/usr/local/lib -R/usr/local/lib -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/bsddb3/_pybsddb.so -ldb-4.3
cc: error: unknown argument: '-R/usr/local/lib'
error: command 'cc' failed with exit status 1


I found the patch below to help, it is for function runtime_library_dir_option in Lib/distutils/unixccompiler.py
-Wl,-rpath works for both gcc and clang on freebsd

--- ./Lib/distutils/unixccompiler.py.orig
+++ ./Lib/distutils/unixccompiler.py
@@ -228,6 +228,8 @@
         if sys.platform[:6] == "darwin":
             # MacOSX's linker doesn't understand the -R flag at all
             return "-L" + dir
+        elif sys.platform[:7] == "freebsd":
+            return "-Wl,-rpath=" + dir
         elif sys.platform[:5] == "hp-ux":
             if self._is_gcc(compiler):
                 return ["-Wl,+s", "-L" + dir]
msg212177 - (view) Author: koobs (koobs) Date: 2014-02-25 13:40
Details on how clang 3.4 changes behaviour for compiler flags: 

http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#new-compiler-flags
msg212185 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-02-25 15:48
this looks safe from my point of view.

However the real problem is that you unconditionally add a runtime path for a standard system path.  I think the better way to fix this is not to pass the -L and -R arguments at all if the library is found in a system path.
msg212188 - (view) Author: Antoine Brodin.FreeBSD (Antoine.Brodin.FreeBSD) Date: 2014-02-25 16:45
For the python-ldap extension, this seems to be a buglet in its setup.cfg, it lists /usr/lib in library_dirs and /usr/include in library_dirs

For the others, /usr/local/lib is not in the default library search path (only /lib and /usr/lib) so at least -L has to be specified.
msg212193 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2014-02-25 18:26
doko: how do you know the addition of the -R option is unconditional? and whom do you refer to by "you" who is adding the option?

In any case, the patch is independent of whether the option is added unconditionally, and I agree that the patch looks safe. The question is whether it should be extended to the other *BSDs as well.
msg212415 - (view) Author: koobs (koobs) Date: 2014-02-28 09:04
Attaching patch against default
msg212419 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-02-28 10:32
>  The question is whether it should be extended to the other *BSDs as well.

The change is not needed on Linux (if Clang 3.4 is used to compile extensions too)?
msg212718 - (view) Author: koobs (koobs) Date: 2014-03-04 13:23
It would be great if this could make it into 3.4, extended to include other OS's if necessary at a later date.
msg212719 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-03-04 13:38
https://bugs.gentoo.org/show_bug.cgi?id=503352 suggests that this problem occurs also on Linux when using Clang.
msg215310 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2014-04-01 13:12
Can we have some more feedback on this?
msg215312 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-04-01 13:16
See also issue #21122.
msg215315 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2014-04-01 13:29
That does look to be a different issue, though.
History
Date User Action Args
2014-04-01 13:29:58djcsetmessages: + msg215315
2014-04-01 13:16:22hayposetmessages: + msg215312
2014-04-01 13:12:52djcsetmessages: + msg215310
2014-04-01 13:12:27djcsetnosy: + djc
2014-03-04 13:38:57Arfreversetmessages: + msg212719
2014-03-04 13:23:03koobssetmessages: + msg212718
2014-02-28 10:32:18hayposetnosy: + haypo
messages: + msg212419
2014-02-28 09:04:49koobssetfiles: + python-issue20767.diff
type: compile error
messages: + msg212415

keywords: + patch
2014-02-26 04:50:02Arfreversetnosy: + Arfrever
2014-02-25 18:26:18loewissetmessages: + msg212193
2014-02-25 16:45:44Antoine.Brodin.FreeBSDsetmessages: + msg212188
2014-02-25 15:48:31dokosetmessages: + msg212185
2014-02-25 15:44:47rhettingersetpriority: normal -> high
2014-02-25 13:40:55koobssetmessages: + msg212177
2014-02-25 13:38:36koobssetversions: + Python 3.3, Python 3.4, Python 3.5
2014-02-25 13:34:27hayposetnosy: + loewis, doko, thomas-petazzoni
2014-02-25 13:33:18Antoine.Brodin.FreeBSDcreate