Title: Python 3.5rc1 compilation error with Apple clang 4.2 included with Xcode 4
Type: compile error Stage: resolved
Components: Versions: Python 3.5
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mojca Miklavec, benjamin.peterson, dabeaz, larry, larryv, ned.deily, ronaldoussoren, tdsmith, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2015-08-11 17:06 by dabeaz, last changed 2021-01-02 02:42 by dabeaz. This issue is now closed.

File name Uploaded Description Edit
python3.5.2-should-not-use-libatomic-with-old-clang.diff Mojca Miklavec, 2016-08-02 14:38 A patch to prevent false detection of functional "atomic" in clang < 3.3 (maybe 3.4)
Messages (7)
msg248415 - (view) Author: David Beazley (dabeaz) Date: 2015-08-11 17:06
Just a note that Python-3.5.0rc1 fails to compile on Mac OS X 10.8.5 with the following compiler:

bash$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

Here is the resulting compilation error:

/usr/bin/clang -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c
fatal error: error in backend: Cannot select: 0x102725710: i8,ch =
      AtomicSwap 0x102c45ce0, 0x102725010, 0x102725510<Volatile
      ST1[@gil_drop_request.0.b]> [ID=7]
  0x102725010: i64 = X86ISD::WrapperRIP 0x102723710 [ID=6]
    0x102723710: i64 = TargetGlobalAddress<i1* @gil_drop_request.0.b> 0 [ID=4]
  0x102725510: i8 = Constant<1> [ID=2]
In function: take_gil
make: *** [Python/ceval.o] Error 1

Problem can be fixed by commenting out the following line in pyconfig.h

/* Has builtin atomics */
// #define HAVE_BUILTIN_ATOMIC 1   

Not really sure what to advise.  To my eyes, it looks like a bug in clang or Xcode.  So, maybe this is more just an FYI that source builds might fail on certain older Mac systems.
msg248431 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-08-11 21:48
This is a regression from previous releases of Python.  It was introduced by fbe87fb071a6 (for Issue22038) which added the use of C built-in functions for atomic memory access for additional architectures like x86_64.  It seems that the relatively early version of Apple's clang included with Xcode 4 fails compiling this code.  The specific clang version that failed ("Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)") is the version included with the last shipped version of Xcode 4, Xcode 4.6.3, for OS X 10.7 and 10.8.  For OS X 10.8, there is available a more recent version of Xcode, Xcode 5.1.1, which includes a new version of clang, "Apple LLVM 5.1 (clang-503.0.40)", which does correctly compile this code.  In general, we always recommend using the most recent available version of Xcode- or Command Line Tools-installed build tools for a particular version of OS X (with the notable exception of OS X 10.6 - stick with Xcode 3.2.6 there).  So upgrading to Xcode 5.1.1 on OS X 10.8.5 should solve the problem there.  Unfortunately, for OS X 10.7.x, Xcode 4.6.3 is the most recent version and Xcode 4 has always been somewhat problematic for Python builds.  It was in Xcode 4 that Apple stopped shipping the "native" gcc-4.2 compiler in favor of clang and the hybrid Apple llvm-gcc-4.2 compiler (which uses gcc as the front end and LLVM as the backend).  The llvm-gcc-4.2 compiler was a transitional tool, not well-maintained, and is known to incorrectly compile recent versions of Python 3 (Issue13241) so it cannot be used as a substitute for clang in this case on OS X 10.7 (it was no longer shipped as of Xcode 5).

Support for OS X 10.8 is much more important than OS X 10.7: I doubt there are *that* many users are still on 10.7.  So I think a case could be made for marking this as "won't fix".  On the other hand, it should be possible to add tests to ./configure to skip trying to use the atomic builtins when this compiler is in use.
msg248432 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-11 21:49
I'd need to see the patch to be certain, but yes my assumption is I'd accept a pull request for this.
msg248488 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-08-12 23:22
You can try to add an ifdef in Include/pyatomic.h.
msg249086 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 00:10
Please either mark as wontfix or send me a pull request.
msg249095 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-08-25 01:23
I don't think we should hold 3.5.0 for a patch for this.  I recommend lowering the priority and targeting a patch for 3.5.1.
msg256214 - (view) Author: Mojca Miklavec (Mojca Miklavec) * Date: 2015-12-11 13:33
Is anyone willing to help me a bit to come up with a pull request?

I took some ideas from here:

Citation from that patch:
   Clang 3.0 has a partial implementation of GNU atomics; 3.1 rounds it out.

   Apple changes the Clang version to track Xcode versions; use
   __apple_build_version__ to distinguish between the two.

   Xcode 4.3 uses Apple LLVM 3.1, which corresponds to Clang 3.1.

One should probably adapt this into "at least clang 3.3 (or perhaps even 3.4?) is needed". Now, the basic test in is the following:

    volatile int val = 1;
    int main() {
        __atomic_load_n(&val, __ATOMIC_SEQ_CST);
        return 0;

With clang on 10.7 (Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)) this compiles, runs and returns 0. Evidently a more complex test is needed to trigger the error, but I'm not sure that that test should be since I don't have enough experience with "atomic". If you have a clue what functionality I should test, I will gladly do it and try to come up with a better test.

An alternative could be to blacklist a compiler and this is what I am proposing here as a very naive approach:

    volatile int val = 1;
    int main() {
        __atomic_load_n(&val, __ATOMIC_SEQ_CST);

    #define VERSION_CHECK(cc_major, cc_minor, req_major, req_minor) \
        ((cc_major) > (req_major) || \
        (cc_major) == (req_major) && (cc_minor) >= (req_minor))

    #if defined(__clang__)
        #if defined(__apple_build_version__)
            // either one test or the other should work
            // #if __apple_build_version__ < 5000000
            #if !VERSION_CHECK(__clang_major__, __clang_minor__, 5, 0)
        // not sure if this is 3.3 or 3.4
        #elif !VERSION_CHECK(__clang_major__, __clang_minor__, 3, 3)

      return 0;
Date User Action Args
2021-01-02 02:42:25dabeazsetstatus: open -> closed
stage: resolved
2016-08-04 17:36:18larryvsetnosy: + larryv
2016-08-02 14:40:19Mojca Miklavecsetfiles: - python3.5.2-should-not-use-libatomic-with-old-clang.diff
2016-08-02 14:38:03Mojca Miklavecsetfiles: + python3.5.2-should-not-use-libatomic-with-old-clang.diff
2016-08-02 14:29:01Mojca Miklavecsetfiles: + python3.5.2-should-not-use-libatomic-with-old-clang.diff
keywords: + patch
2015-12-27 05:44:43tdsmithsetnosy: + tdsmith
2015-12-11 13:33:07Mojca Miklavecsetnosy: + Mojca Miklavec
messages: + msg256214
2015-08-25 01:23:31ned.deilysetpriority: release blocker -> normal

messages: + msg249095
2015-08-25 00:10:43larrysetmessages: + msg249086
2015-08-14 14:17:58ronaldoussorensetnosy: + ronaldoussoren
2015-08-12 23:22:45vstinnersetmessages: + msg248488
2015-08-11 21:49:14larrysetmessages: + msg248432
2015-08-11 21:48:07ned.deilysetnosy: + vstinner, larry

messages: + msg248431
title: Python 3.5rc1 compilation error on OS X 10.8 -> Python 3.5rc1 compilation error with Apple clang 4.2 included with Xcode 4
2015-08-11 17:12:07yselivanovsetpriority: normal -> release blocker
nosy: + benjamin.peterson, ned.deily, yselivanov
2015-08-11 17:06:44dabeazcreate