Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some python extensions can't be compiled with clang 3.4 #64966

Closed
AntoineBrodinFreeBSD mannequin opened this issue Feb 25, 2014 · 33 comments
Closed

Some python extensions can't be compiled with clang 3.4 #64966

AntoineBrodinFreeBSD mannequin opened this issue Feb 25, 2014 · 33 comments
Labels
build The build process and cross-build easy extension-modules C modules in the Modules dir

Comments

@AntoineBrodinFreeBSD
Copy link
Mannequin

AntoineBrodinFreeBSD mannequin commented Feb 25, 2014

BPO 20767
Nosy @loewis, @warsaw, @doko42, @pfmoore, @vstinner, @larryhastings, @tjguk, @djc, @ezio-melotti, @merwok, @asvetlov, @skrah, @zware, @1st1, @koobs, @zooba, @dstufft, @matrixise, @tpetazzoni, @moreati, @evelynmitchell
Files
  • python-issue20767.diff
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-01-31.10:26:46.865>
created_at = <Date 2014-02-25.13:33:18.121>
labels = ['extension-modules', 'easy', 'build']
title = "Some python extensions can't be compiled with clang 3.4"
updated_at = <Date 2019-01-31.22:45:10.609>
user = 'https://bugs.python.org/AntoineBrodinFreeBSD'

    bugs.python.org fields:

    activity = <Date 2019-01-31.22:45:10.609>
actor = 'r.david.murray'
assignee = 'docs@python'
closed = True
closed_date = <Date 2019-01-31.10:26:46.865>
closer = 'vstinner'
components = ['Extension Modules']
creation = <Date 2014-02-25.13:33:18.121>
creator = 'Antoine.Brodin.FreeBSD'
dependencies = []
files = ['34253']
hgrepos = []
issue_num = 20767
keywords = ['patch', 'easy', 'needs review']
message_count = 33.0
messages = ['212176', '212177', '212185', '212188', '212193', '212415', '212419', '212718', '212719', '215310', '215312', '215315', '257021', '270077', '270767', '271845', '271850', '271851', '271852', '271856', '271857', '271883', '271884', '271885', '271889', '271890', '281845', '310437', '310446', '311315', '311316', '334606', '334620']
nosy_count = 26.0
nosy_names = ['loewis', 'barry', 'doko', 'paul.moore', 'vstinner', 'larry', 'tim.golden', 'djc', 'ezio.melotti', 'eric.araujo', 'mrabarnett', 'Arfrever', 'asvetlov', 'skrah', 'docs@python', 'python-dev', 'zach.ware', 'yselivanov', 'koobs', 'steve.dower', 'dstufft', 'matrixise', 'thomas-petazzoni', 'Antoine.Brodin.FreeBSD', 'Alex.Willmer', 'Evelyn Mitchell']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue20767'
versions = ['Python 2.7', 'Python 3.6']
    @AntoineBrodinFreeBSD
    Copy link
    Mannequin Author

    AntoineBrodinFreeBSD mannequin commented Feb 25, 2014

    Hi,

    On FreeBSD -current, clang 3.4 is now the default compiler.
    Clang 3.4 rejects "-R/path/to/lib" flag (previously in version 3.3 it just ignored it).

    This leads to some errors with some python extensions:

    cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cache.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/connection.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/cursor.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/microprotocols.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/module.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/prepare_protocol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/row.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/statement.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/_sqlite/util.o -L/usr/local/lib -R/usr/local/lib -lsqlite3 -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/_sqlite3.so
    cc: error: unknown argument: '-R/usr/local/lib'
    error: command 'cc' failed with exit status 1

    cc -shared -O2 -pipe -DLDAP_DEPRECATED -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/LDAPObject.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapcontrol.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/common.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/constants.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/errors.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/functions.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/schema.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/ldapmodule.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/message.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/version.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/options.o build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/berval.o -L/usr/local/lib -L/usr/lib -R/usr/local/lib -R/usr/lib -lldap_r -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/_ldap.so
    cc: error: unknown argument: '-R/usr/local/lib'
    cc: error: unknown argument: '-R/usr/lib'
    error: command 'cc' failed with exit status 1

    cc -shared -O2 -pipe -fno-strict-aliasing build/temp.freebsd-11.0-CURRENT-amd64-2.7/Modules/_bsddb.o -L/usr/local/lib -R/usr/local/lib -o build/lib.freebsd-11.0-CURRENT-amd64-2.7/bsddb3/_pybsddb.so -ldb-4.3
    cc: error: unknown argument: '-R/usr/local/lib'
    error: command 'cc' failed with exit status 1

    I found the patch below to help, it is for function runtime_library_dir_option in Lib/distutils/unixccompiler.py
    -Wl,-rpath works for both gcc and clang on freebsd

    --- ./Lib/distutils/unixccompiler.py.orig
+++ ./Lib/distutils/unixccompiler.py
@@ -228,6 +228,8 @@
         if sys.platform[:6] == "darwin":
             # MacOSX's linker doesn't understand the -R flag at all
             return "-L" + dir
+        elif sys.platform[:7] == "freebsd":
+            return "-Wl,-rpath=" + dir
         elif sys.platform[:5] == "hp-ux":
             if self._is_gcc(compiler):
                 return ["-Wl,+s", "-L" + dir]

    @AntoineBrodinFreeBSD AntoineBrodinFreeBSD mannequin added the stdlib Python modules in the Lib dir label Feb 25, 2014
    @koobs
    Copy link

    koobs commented Feb 25, 2014

    Details on how clang 3.4 changes behaviour for compiler flags:

    http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html#new-compiler-flags

    @doko42
    Copy link
    Member

    doko42 commented Feb 25, 2014

    this looks safe from my point of view.

    However the real problem is that you unconditionally add a runtime path for a standard system path. I think the better way to fix this is not to pass the -L and -R arguments at all if the library is found in a system path.

    @AntoineBrodinFreeBSD
    Copy link
    Mannequin Author

    AntoineBrodinFreeBSD mannequin commented Feb 25, 2014

    For the python-ldap extension, this seems to be a buglet in its setup.cfg, it lists /usr/lib in library_dirs and /usr/include in library_dirs

    For the others, /usr/local/lib is not in the default library search path (only /lib and /usr/lib) so at least -L has to be specified.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Feb 25, 2014

    doko: how do you know the addition of the -R option is unconditional? and whom do you refer to by "you" who is adding the option?

    In any case, the patch is independent of whether the option is added unconditionally, and I agree that the patch looks safe. The question is whether it should be extended to the other *BSDs as well.

    @koobs
    Copy link

    koobs commented Feb 28, 2014

    Attaching patch against default

    @koobs koobs added the build The build process and cross-build label Feb 28, 2014
    @vstinner
    Copy link
    Member

    The question is whether it should be extended to the other *BSDs as well.

    The change is not needed on Linux (if Clang 3.4 is used to compile extensions too)?

    @koobs
    Copy link

    koobs commented Mar 4, 2014

    It would be great if this could make it into 3.4, extended to include other OS's if necessary at a later date.

    @Arfrever
    Copy link
    Mannequin

    Arfrever mannequin commented Mar 4, 2014

    https://bugs.gentoo.org/show_bug.cgi?id=503352 suggests that this problem occurs also on Linux when using Clang.

    @djc
    Copy link
    Member

    djc commented Apr 1, 2014

    Can we have some more feedback on this?

    @vstinner
    Copy link
    Member

    vstinner commented Apr 1, 2014

    See also issue bpo-21122.

    @djc
    Copy link
    Member

    djc commented Apr 1, 2014

    That does look to be a different issue, though.

    @koobs koobs added the easy label Dec 26, 2015
    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Dec 26, 2015

    Over at the llvm bug tracker this is marked as a release blocker:

    https://llvm.org/bugs/show_bug.cgi?id=18164

    @supriyantomaftuh supriyantomaftuh mannequin added build The build process and cross-build extension-modules C modules in the Modules dir interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-tkinter topic-unicode OS-windows topic-XML labels Apr 12, 2016
    @berkerpeksag berkerpeksag removed build The build process and cross-build extension-modules C modules in the Modules dir interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-tkinter topic-unicode OS-windows topic-XML labels Apr 12, 2016
    @PascalvanderDonck PascalvanderDonck mannequin added topic-SSL performance Performance or resource usage and removed build The build process and cross-build labels Jan 31, 2019
    @larryhastings
    Copy link
    Contributor

    It's too late to fix this for Python 3.4 and 3.5, as those are now in security-fixes-only mode. Also, please don't select every possible component that could be remotely connected.

    @larryhastings larryhastings added build The build process and cross-build and removed build The build process and cross-build stdlib Python modules in the Lib dir docs Documentation in the Doc dir topic-IDLE topic-installation interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-regex tests Tests in the Lib/test dir topic-tkinter topic-unicode topic-XML topic-2to3 topic-ctypes topic-IO topic-email topic-asyncio topic-argument-clinic topic-SSL performance Performance or resource usage labels Jan 31, 2019
    @vstinner
    Copy link
    Member

    If the intent is to use this issue to track the resolution of the original issue (clang compilation broken) and cover the new PR (which just moves the code from os detection to compiler detection), then it can remain open. Otherwise it can be closed, as the original issue 'has' been fixed.

    Ok, I close the issue.

    On #5233 bapt asked "code changes", but I don't understand what exactly and I'm not interested to continue to work on these changes. If someone is interested, please open a new issue (pointing to this one).

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    build The build process and cross-build easy extension-modules C modules in the Modules dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    7 participants
    @berkerpeksag @matrixise @djc @vstinner @koobs @doko42 @larryhastings