This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch
Type: compile error Stage: resolved
Components: Build Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: debohman, indygreg, lukasz.langa, miss-islington, ned.deily
Priority: Keywords: patch

Created on 2021-10-07 12:56 by debohman, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28845 merged debohman, 2021-10-10 01:36
PR 28910 merged miss-islington, 2021-10-13 00:11
PR 28911 merged miss-islington, 2021-10-13 00:11
PR 31889 merged ned.deily, 2022-03-15 07:00
PR 31890 merged ned.deily, 2022-03-15 07:02
Messages (24)
msg403401 - (view) Author: David Bohman (debohman) * Date: 2021-10-07 12:56
% CC=clang CXX=clang++ ./configure --enable-optimizations --with-lto
checking build system type... x86_64-apple-darwin16.7.0
checking host system type... x86_64-apple-darwin16.7.0
checking for python3.10... no
checking for python3... python3
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "darwin"
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for a sed that does not truncate output... /usr/local/bin/sed
checking for --with-cxx-main=<compiler>... no
checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report

The problem occurs here at line 5382 of configure:

if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then
  if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
    as_fn_error $? "internal configure error for the platform triplet, please file a bug report" "$LINENO" 5
  fi

The problem is that $PLATFORM_TRIPLET is set to darwin (which appears to be correct) and $MULTIARCH is set thus:

MULTIARCH=$($CC --print-multiarch 2>/dev/null)

which evaluates to x86_64-apple-darwin16.7.0.

This is with the public llvm / clang version 13.0.0.

If you set MULTIARCH=darwin in configure, python configures, builds and runs.
msg403424 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-07 17:31
On what platform and OS version are you building? When you say "public llvm / clang", do you mean from Apple for macOS or from some other source? Be aware that we only officially test and support building Python for macOS on macOS itself using the Apple build tools (Xcode or Command Line Tools).
msg403443 - (view) Author: David Bohman (debohman) * Date: 2021-10-07 21:31
By "public llvm / clang", I mean the toolchain version released by the llvm project.

They just released version 13.0.0 last week.

The problem is that version 12 of llvm / clang did not implement --print-multiarch, so this logic in configure was not exercised due to:

if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then

failing. In that case, $MULTIARCH would be a null string, so test x$MULTIARCH != x would fail. It seems that the logic in configure needs to be updated so that it functions correctly on darwin where --print-multiarch is implemented.
msg403454 - (view) Author: David Bohman (debohman) * Date: 2021-10-08 02:11
I can put together a pull request to fix this. The fix is in configure.ac.

Should I include the mods from the autoreconf in the PR?
msg403455 - (view) Author: David Bohman (debohman) * Date: 2021-10-08 02:32
Also, what branch should the PR be on, 3.10 or main?
msg403456 - (view) Author: David Bohman (debohman) * Date: 2021-10-08 03:20
Okay, I see from the developer guide, that the generated files should be included. Is it permissible to use the current version of autoconf?
msg403515 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-09 03:21
OK, I've had a chance to look at this issue and I now understand what you were describing. So, yes, it would be good to fix the test in configure so that it does work with a newer clang that does implement the --print-multiarch command line option. FTR, this is not an issue with clang from any Apple tools chains (Xcode or Command Line Tools) released to this point. A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary. We currently use autoconf 2.69 but, if that is not available, a core developer can regenerate configure prior to merging.
msg403556 - (view) Author: David Bohman (debohman) * Date: 2021-10-09 22:25
Yes, I don't understand why the llvm / clang decided to implement this in version 13. In version 12, the switch caused an error to stderr, and nothing to stdout. GCC 11.1 generates a single \n to stdout.

Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines? Early on, I tried setting $PLATFORM_TRIPLET in the script to the actual value returned by the complier, and that blew up later during the actual build. That was when I decided to just fix the logic so that it continued to operate the same on all platforms, even darwin with --print-multiarch generating the triplet.
msg403558 - (view) Author: David Bohman (debohman) * Date: 2021-10-09 22:42
I have version 2.71 of autoconf and the latest autoconf-archive installed on my machine.
msg403561 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-10 00:28
> Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines?

For now, yes. Apple long ago implemented the equivalent of multi-architecture support using "-arch" in the compiler driver and the Python build system (configure, Makefile, setup.py, distutils, et al) supports that for macOS without using PLATFORM_TRIPLET.
msg403562 - (view) Author: David Bohman (debohman) * Date: 2021-10-10 00:32
Okay. Should the branch for the PR be off the main branch, or 3.10?
msg403563 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-10 00:34
"A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary."
msg403682 - (view) Author: Gregory Szorc (indygreg) * Date: 2021-10-11 20:35
Note that this issue isn't macOS specific: you will get the same failure when cross-compiling targeting Linux. e.g. --build=x86_64-unknown-linux-gnu --host=i686-unknown-linux-gnu.
msg403685 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-11 20:56
> Note that this issue isn't macOS specific: you will get the same failure when cross-compiling targeting Linux. e.g. --build=x86_64-unknown-linux-gnu --host=i686-unknown-linux-gnu.

Can you be more specific? This particular issue isn't about cross-compiling. It's just about building for macOS on macOS. Under what circumstances is there a problem when trying to build one of the supported cross-compilation targets and does this proposed change fix that problem or create a new problem?
msg403767 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-13 00:10
New changeset 9c4766772cda67648184f8ddba546a5fc0167f91 by David Bohman in branch 'main':
bpo-45405: Prevent ``internal configure error`` when running ``configure``  with recent versions of non-Apple clang. (#28845)
https://github.com/python/cpython/commit/9c4766772cda67648184f8ddba546a5fc0167f91
msg403769 - (view) Author: miss-islington (miss-islington) Date: 2021-10-13 00:31
New changeset edae3e2ac73151217b4b4e096dd9f17cc0a50c11 by Miss Islington (bot) in branch '3.10':
[3.10] bpo-45405: Prevent ``internal configure error`` when running ``configure``  with recent versions of non-Apple clang. (GH-28845) (GH-28911)
https://github.com/python/cpython/commit/edae3e2ac73151217b4b4e096dd9f17cc0a50c11
msg403771 - (view) Author: miss-islington (miss-islington) Date: 2021-10-13 00:36
New changeset 9901d153c201d852d27dc9d3074e283c26468f6d by Miss Islington (bot) in branch '3.9':
[3.9] bpo-45405: Prevent ``internal configure error`` when running ``configure``  with recent versions of non-Apple clang. (GH-28845) (GH-28910)
https://github.com/python/cpython/commit/9901d153c201d852d27dc9d3074e283c26468f6d
msg403773 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-13 00:52
Merged for release in 3.10.1 and 3.9.8. Thanks for the issue and PR, David!
msg403839 - (view) Author: David Bohman (debohman) * Date: 2021-10-13 14:49
Thank you for your help, Ned.
msg403881 - (view) Author: Gregory Szorc (indygreg) * Date: 2021-10-14 02:29
> Can you be more specific? This particular issue isn't about cross-compiling.

The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

I was able to perturb this same error on Linux with an x86_64 Clang 13 with configure options --build=x86_64-unknown-linux-gnu --host=i686-unknown-linux-gnu. https://github.com/indygreg/python-build-standalone/runs/3853832124?check_suite_focus=true should work if you are logged into GitHub. Not the exact same failure, but related since it involves --print-multiarch suddenly working.
msg403886 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-10-14 04:54
> The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

If there are still issues on other supported platforms, PRs are welcome.
msg415220 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2022-03-15 07:16
As of Xcode 13.3, released 2022-03-14 to support macOS 12.3 et al, the included Apple Clang (Apple clang version 13.1.6 (clang-1316.0.21.2)) now supports the  --print-multiarch option and so Python 3.8 and 3.7 are now vulnerable to this issue when building with that version of Xcode or Apple Command Line Tools on macOS. While both 3.8 and 3.7 are now in the security-fix-only phase of their life cycles and are not fully supported on macOS 12, you should still be able to run ./configure without error using the current Apple Developer Tools. PR 31889 and PR 31890 are backports of this to 3.8 and 3.7. Setting to "release blocker" priority for upcoming 3.8 and 3.7 releases.
msg415221 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2022-03-15 07:18
New changeset 720bb456dc711b0776bae837d1f9a0b10c28ddf2 by Ned Deily in branch '3.7':
bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31890)
https://github.com/python/cpython/commit/720bb456dc711b0776bae837d1f9a0b10c28ddf2
msg415247 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2022-03-15 14:39
New changeset dea270a2a80214de22afadaaca2043d0d782eb7d by Ned Deily in branch '3.8':
bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31889)
https://github.com/python/cpython/commit/dea270a2a80214de22afadaaca2043d0d782eb7d
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89568
2022-03-15 14:47:54ned.deilysetpriority: release blocker ->
status: open -> closed
resolution: fixed
stage: commit review -> resolved
2022-03-15 14:39:24lukasz.langasetmessages: + msg415247
2022-03-15 07:18:42ned.deilysetmessages: + msg415221
2022-03-15 07:16:04ned.deilysetstatus: closed -> open
priority: normal -> release blocker

versions: + Python 3.7, Python 3.8, - Python 3.9, Python 3.10, Python 3.11
nosy: + lukasz.langa

messages: + msg415220
resolution: fixed -> (no value)
stage: resolved -> commit review
2022-03-15 07:02:59ned.deilysetpull_requests: + pull_request29988
2022-03-15 07:00:47ned.deilysetpull_requests: + pull_request29987
2021-10-14 04:54:00ned.deilysetmessages: + msg403886
2021-10-14 02:29:46indygregsetmessages: + msg403881
2021-10-13 14:49:15debohmansetmessages: + msg403839
2021-10-13 00:52:47ned.deilysetstatus: open -> closed
resolution: fixed
messages: + msg403773

stage: patch review -> resolved
2021-10-13 00:36:06miss-islingtonsetmessages: + msg403771
2021-10-13 00:31:39miss-islingtonsetmessages: + msg403769
2021-10-13 00:11:20miss-islingtonsetpull_requests: + pull_request27201
2021-10-13 00:11:16miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27200
2021-10-13 00:10:33ned.deilysetmessages: + msg403767
2021-10-11 20:56:48ned.deilysetmessages: + msg403685
2021-10-11 20:35:40indygregsetnosy: + indygreg
messages: + msg403682
2021-10-10 01:36:58debohmansetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request27157
2021-10-10 00:34:07ned.deilysetmessages: + msg403563
2021-10-10 00:32:37debohmansetmessages: + msg403562
2021-10-10 00:28:38ned.deilysetmessages: + msg403561
2021-10-09 22:42:05debohmansetmessages: + msg403558
2021-10-09 22:25:48debohmansetmessages: + msg403556
2021-10-09 03:21:59ned.deilysettitle: Python does not configure on darwin using public llvm / clang version 13.0.0 -> configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch
stage: needs patch
messages: + msg403515
versions: + Python 3.9, Python 3.10, Python 3.11
2021-10-08 03:20:26debohmansetmessages: + msg403456
2021-10-08 02:33:07debohmansetversions: - Python 3.10
title: Python 3.10.0 does not configure on darwin using public llvm / clang -> Python does not configure on darwin using public llvm / clang version 13.0.0
2021-10-08 02:32:24debohmansetmessages: + msg403455
2021-10-08 02:11:05debohmansetmessages: + msg403454
2021-10-07 21:31:12debohmansetmessages: + msg403443
2021-10-07 17:31:11ned.deilysetnosy: + ned.deily
messages: + msg403424
2021-10-07 12:56:37debohmancreate