classification
Title: Python 3.5rc1 compilation error with Apple clang 4.2 included with Xcode 4
Type: compile error Stage: resolved
Components: Versions: Python 3.5
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mojca Miklavec, benjamin.peterson, dabeaz, larry, larryv, ned.deily, ronaldoussoren, tdsmith, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2015-08-11 17:06 by dabeaz, last changed 2021-01-02 02:42 by dabeaz. This issue is now closed.

Files
File name Uploaded Description Edit
python3.5.2-should-not-use-libatomic-with-old-clang.diff Mojca Miklavec, 2016-08-02 14:38 A patch to prevent false detection of functional "atomic" in clang < 3.3 (maybe 3.4)
Messages (7)
msg248415 - (view) Author: David Beazley (dabeaz) Date: 2015-08-11 17:06
Just a note that Python-3.5.0rc1 fails to compile on Mac OS X 10.8.5 with the following compiler:

bash$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix
bash$ 

Here is the resulting compilation error:

/usr/bin/clang -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes    -Werror=declaration-after-statement   -I. -IInclude -I./Include    -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c
fatal error: error in backend: Cannot select: 0x102725710: i8,ch =
      AtomicSwap 0x102c45ce0, 0x102725010, 0x102725510<Volatile
      ST1[@gil_drop_request.0.b]> [ID=7]
  0x102725010: i64 = X86ISD::WrapperRIP 0x102723710 [ID=6]
    0x102723710: i64 = TargetGlobalAddress<i1* @gil_drop_request.0.b> 0 [ID=4]
  0x102725510: i8 = Constant<1> [ID=2]
In function: take_gil
make: *** [Python/ceval.o] Error 1

Problem can be fixed by commenting out the following line in pyconfig.h

/* Has builtin atomics */
// #define HAVE_BUILTIN_ATOMIC 1   


Not really sure what to advise.  To my eyes, it looks like a bug in clang or Xcode.  So, maybe this is more just an FYI that source builds might fail on certain older Mac systems.
msg248431 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-08-11 21:48
This is a regression from previous releases of Python.  It was introduced by fbe87fb071a6 (for Issue22038) which added the use of C built-in functions for atomic memory access for additional architectures like x86_64.  It seems that the relatively early version of Apple's clang included with Xcode 4 fails compiling this code.  The specific clang version that failed ("Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)") is the version included with the last shipped version of Xcode 4, Xcode 4.6.3, for OS X 10.7 and 10.8.  For OS X 10.8, there is available a more recent version of Xcode, Xcode 5.1.1, which includes a new version of clang, "Apple LLVM 5.1 (clang-503.0.40)", which does correctly compile this code.  In general, we always recommend using the most recent available version of Xcode- or Command Line Tools-installed build tools for a particular version of OS X (with the notable exception of OS X 10.6 - stick with Xcode 3.2.6 there).  So upgrading to Xcode 5.1.1 on OS X 10.8.5 should solve the problem there.  Unfortunately, for OS X 10.7.x, Xcode 4.6.3 is the most recent version and Xcode 4 has always been somewhat problematic for Python builds.  It was in Xcode 4 that Apple stopped shipping the "native" gcc-4.2 compiler in favor of clang and the hybrid Apple llvm-gcc-4.2 compiler (which uses gcc as the front end and LLVM as the backend).  The llvm-gcc-4.2 compiler was a transitional tool, not well-maintained, and is known to incorrectly compile recent versions of Python 3 (Issue13241) so it cannot be used as a substitute for clang in this case on OS X 10.7 (it was no longer shipped as of Xcode 5).

Support for OS X 10.8 is much more important than OS X 10.7: I doubt there are *that* many users are still on 10.7.  So I think a case could be made for marking this as "won't fix".  On the other hand, it should be possible to add tests to ./configure to skip trying to use the atomic builtins when this compiler is in use.
msg248432 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-11 21:49
I'd need to see the patch to be certain, but yes my assumption is I'd accept a pull request for this.
msg248488 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-08-12 23:22
You can try to add an ifdef in Include/pyatomic.h.
msg249086 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 00:10
Please either mark as wontfix or send me a pull request.
msg249095 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-08-25 01:23
I don't think we should hold 3.5.0 for a patch for this.  I recommend lowering the priority and targeting a patch for 3.5.1.
msg256214 - (view) Author: Mojca Miklavec (Mojca Miklavec) * Date: 2015-12-11 13:33
Is anyone willing to help me a bit to come up with a pull request?

I took some ideas from here:
https://fossies.org/diffs/xscreensaver/5.30_vs_5.32/utils/thread_util.c-diff.html

Citation from that patch:
   Clang 3.0 has a partial implementation of GNU atomics; 3.1 rounds it out.

   http://llvm.org/viewvc/llvm-project/cfe/tags/RELEASE_30/final/include/clang/Basic/Builtins.def?view=markup
   http://llvm.org/viewvc/llvm-project/cfe/tags/RELEASE_31/final/include/clang/Basic/Builtins.def?view=markup

   Apple changes the Clang version to track Xcode versions; use
   __apple_build_version__ to distinguish between the two.

   Xcode 4.3 uses Apple LLVM 3.1, which corresponds to Clang 3.1.
   https://en.wikipedia.org/wiki/Xcode

One should probably adapt this into "at least clang 3.3 (or perhaps even 3.4?) is needed". Now, the basic test in configure.ac is the following:

    volatile int val = 1;
    int main() {
        __atomic_load_n(&val, __ATOMIC_SEQ_CST);
        return 0;
    }

With clang on 10.7 (Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)) this compiles, runs and returns 0. Evidently a more complex test is needed to trigger the error, but I'm not sure that that test should be since I don't have enough experience with "atomic". If you have a clue what functionality I should test, I will gladly do it and try to come up with a better test.

An alternative could be to blacklist a compiler and this is what I am proposing here as a very naive approach:

    volatile int val = 1;
    int main() {
 
        __atomic_load_n(&val, __ATOMIC_SEQ_CST);

    #define VERSION_CHECK(cc_major, cc_minor, req_major, req_minor) \
        ((cc_major) > (req_major) || \
        (cc_major) == (req_major) && (cc_minor) >= (req_minor))

    #if defined(__clang__)
        #if defined(__apple_build_version__)
            // either one test or the other should work
            // #if __apple_build_version__ < 5000000
            #if !VERSION_CHECK(__clang_major__, __clang_minor__, 5, 0)
                this_should_fail();
            #endif
        // not sure if this is 3.3 or 3.4
        #elif !VERSION_CHECK(__clang_major__, __clang_minor__, 3, 3)
            this_should_fail();
        #endif
    #endif

      return 0;
    }
History
Date User Action Args
2021-01-02 02:42:25dabeazsetstatus: open -> closed
stage: resolved
2016-08-04 17:36:18larryvsetnosy: + larryv
2016-08-02 14:40:19Mojca Miklavecsetfiles: - python3.5.2-should-not-use-libatomic-with-old-clang.diff
2016-08-02 14:38:03Mojca Miklavecsetfiles: + python3.5.2-should-not-use-libatomic-with-old-clang.diff
2016-08-02 14:29:01Mojca Miklavecsetfiles: + python3.5.2-should-not-use-libatomic-with-old-clang.diff
keywords: + patch
2015-12-27 05:44:43tdsmithsetnosy: + tdsmith
2015-12-11 13:33:07Mojca Miklavecsetnosy: + Mojca Miklavec
messages: + msg256214
2015-08-25 01:23:31ned.deilysetpriority: release blocker -> normal

messages: + msg249095
2015-08-25 00:10:43larrysetmessages: + msg249086
2015-08-14 14:17:58ronaldoussorensetnosy: + ronaldoussoren
2015-08-12 23:22:45vstinnersetmessages: + msg248488
2015-08-11 21:49:14larrysetmessages: + msg248432
2015-08-11 21:48:07ned.deilysetnosy: + vstinner, larry

messages: + msg248431
title: Python 3.5rc1 compilation error on OS X 10.8 -> Python 3.5rc1 compilation error with Apple clang 4.2 included with Xcode 4
2015-08-11 17:12:07yselivanovsetpriority: normal -> release blocker
nosy: + benjamin.peterson, ned.deily, yselivanov
2015-08-11 17:06:44dabeazcreate