Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch #89568

Closed
debohman mannequin opened this issue Oct 7, 2021 · 24 comments
Closed

configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch #89568

debohman mannequin opened this issue Oct 7, 2021 · 24 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes build The build process and cross-build

Comments

@debohman
Copy link
Mannequin

debohman mannequin commented Oct 7, 2021

BPO 45405
Nosy @ned-deily, @ambv, @indygreg, @debohman, @miss-islington
PRs
  • bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. #28845
  • [3.9] bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. (GH-28845) #28910
  • [3.10] bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. (GH-28845) #28911
  • [3.8] bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) #31889
  • [3.7] bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) #31890
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-03-15.14:47:54.189>
created_at = <Date 2021-10-07.12:56:37.730>
labels = ['3.8', 'build', '3.7']
title = 'configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch'
updated_at = <Date 2022-03-15.14:47:54.189>
user = 'https://github.com/debohman'

    bugs.python.org fields:

    activity = <Date 2022-03-15.14:47:54.189>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-15.14:47:54.189>
closer = 'ned.deily'
components = ['Build']
creation = <Date 2021-10-07.12:56:37.730>
creator = 'debohman'
dependencies = []
files = []
hgrepos = []
issue_num = 45405
keywords = ['patch']
message_count = 24.0
messages = ['403401', '403424', '403443', '403454', '403455', '403456', '403515', '403556', '403558', '403561', '403562', '403563', '403682', '403685', '403767', '403769', '403771', '403773', '403839', '403881', '403886', '415220', '415221', '415247']
nosy_count = 5.0
nosy_names = ['ned.deily', 'lukasz.langa', 'indygreg', 'debohman', 'miss-islington']
pr_nums = ['28845', '28910', '28911', '31889', '31890']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue45405'
versions = ['Python 3.7', 'Python 3.8']
    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 7, 2021

    % CC=clang CXX=clang++ ./configure --enable-optimizations --with-lto
    checking build system type... x86_64-apple-darwin16.7.0
    checking host system type... x86_64-apple-darwin16.7.0
    checking for python3.10... no
    checking for python3... python3
    checking for --enable-universalsdk... no
    checking for --with-universal-archs... no
    checking MACHDEP... "darwin"
    checking for gcc... clang
    checking whether the C compiler works... yes
    checking for C compiler default output file name... a.out
    checking for suffix of executables...
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether we are using the GNU C compiler... yes
    checking whether clang accepts -g... yes
    checking for clang option to accept ISO C89... none needed
    checking how to run the C preprocessor... clang -E
    checking for grep that handles long lines and -e... /usr/bin/grep
    checking for a sed that does not truncate output... /usr/local/bin/sed
    checking for --with-cxx-main=<compiler>... no
    checking for the platform triplet based on compiler characteristics... darwin
    configure: error: internal configure error for the platform triplet, please file a bug report

    The problem occurs here at line 5382 of configure:

    if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then
    if test x$PLATFORM_TRIPLET != x$MULTIARCH; then
    as_fn_error $? "internal configure error for the platform triplet, please file a bug report" "$LINENO" 5
    fi

    The problem is that $PLATFORM_TRIPLET is set to darwin (which appears to be correct) and $MULTIARCH is set thus:

    MULTIARCH=$($CC --print-multiarch 2>/dev/null)

    which evaluates to x86_64-apple-darwin16.7.0.

    This is with the public llvm / clang version 13.0.0.

    If you set MULTIARCH=darwin in configure, python configures, builds and runs.

    @debohman debohman mannequin added 3.10 only security fixes build The build process and cross-build labels Oct 7, 2021
    @ned-deily
    Copy link
    Member

    On what platform and OS version are you building? When you say "public llvm / clang", do you mean from Apple for macOS or from some other source? Be aware that we only officially test and support building Python for macOS on macOS itself using the Apple build tools (Xcode or Command Line Tools).

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 7, 2021

    By "public llvm / clang", I mean the toolchain version released by the llvm project.

    They just released version 13.0.0 last week.

    The problem is that version 12 of llvm / clang did not implement --print-multiarch, so this logic in configure was not exercised due to:

    if test x$PLATFORM_TRIPLET != x && test x$MULTIARCH != x; then

    failing. In that case, $MULTIARCH would be a null string, so test x$MULTIARCH != x would fail. It seems that the logic in configure needs to be updated so that it functions correctly on darwin where --print-multiarch is implemented.

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 8, 2021

    I can put together a pull request to fix this. The fix is in configure.ac.

    Should I include the mods from the autoreconf in the PR?

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 8, 2021

    Also, what branch should the PR be on, 3.10 or main?

    @debohman debohman mannequin removed the 3.10 only security fixes label Oct 8, 2021
    @debohman debohman mannequin changed the title Python 3.10.0 does not configure on darwin using public llvm / clang Python does not configure on darwin using public llvm / clang version 13.0.0 Oct 8, 2021
    @debohman debohman mannequin removed the 3.10 only security fixes label Oct 8, 2021
    @debohman debohman mannequin changed the title Python 3.10.0 does not configure on darwin using public llvm / clang Python does not configure on darwin using public llvm / clang version 13.0.0 Oct 8, 2021
    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 8, 2021

    Okay, I see from the developer guide, that the generated files should be included. Is it permissible to use the current version of autoconf?

    @ned-deily
    Copy link
    Member

    OK, I've had a chance to look at this issue and I now understand what you were describing. So, yes, it would be good to fix the test in configure so that it does work with a newer clang that does implement the --print-multiarch command line option. FTR, this is not an issue with clang from any Apple tools chains (Xcode or Command Line Tools) released to this point. A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary. We currently use autoconf 2.69 but, if that is not available, a core developer can regenerate configure prior to merging.

    @ned-deily ned-deily added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Oct 9, 2021
    @ned-deily ned-deily changed the title Python does not configure on darwin using public llvm / clang version 13.0.0 configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch Oct 9, 2021
    @ned-deily ned-deily added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Oct 9, 2021
    @ned-deily ned-deily changed the title Python does not configure on darwin using public llvm / clang version 13.0.0 configure fails on macOS with non-Apple clang version 13 which implements --print-multiarch Oct 9, 2021
    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 9, 2021

    Yes, I don't understand why the llvm / clang decided to implement this in version 13. In version 12, the switch caused an error to stderr, and nothing to stdout. GCC 11.1 generates a single \n to stdout.

    Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines? Early on, I tried setting $PLATFORM_TRIPLET in the script to the actual value returned by the complier, and that blew up later during the actual build. That was when I decided to just fix the logic so that it continued to operate the same on all platforms, even darwin with --print-multiarch generating the triplet.

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 9, 2021

    I have version 2.71 of autoconf and the latest autoconf-archive installed on my machine.

    @ned-deily
    Copy link
    Member

    Do you plan to stick with $PLATFORM_TRIPLET = darwin, even on the arm machines?

    For now, yes. Apple long ago implemented the equivalent of multi-architecture support using "-arch" in the compiler driver and the Python build system (configure, Makefile, setup.py, distutils, et al) supports that for macOS without using PLATFORM_TRIPLET.

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 10, 2021

    Okay. Should the branch for the PR be off the main branch, or 3.10?

    @ned-deily
    Copy link
    Member

    "A PR against configure.ac in main would be welcome, we can auto-backport to other active branches as necessary."

    @ned-deily
    Copy link
    Member

    Merged for release in 3.10.1 and 3.9.8. Thanks for the issue and PR, David!

    @debohman
    Copy link
    Mannequin Author

    debohman mannequin commented Oct 13, 2021

    Thank you for your help, Ned.

    @indygreg
    Copy link
    Mannequin

    indygreg mannequin commented Oct 14, 2021

    Can you be more specific? This particular issue isn't about cross-compiling.

    The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

    I was able to perturb this same error on Linux with an x86_64 Clang 13 with configure options --build=x86_64-unknown-linux-gnu --host=i686-unknown-linux-gnu. https://github.com/indygreg/python-build-standalone/runs/3853832124?check_suite_focus=true should work if you are logged into GitHub. Not the exact same failure, but related since it involves --print-multiarch suddenly working.

    @ned-deily
    Copy link
    Member

    The point I was trying to make is that clang 13 generically implemented --print-multiarch and the triple logic in configure related to MULTIARCH applies to all platforms, not just macOS.

    If there are still issues on other supported platforms, PRs are welcome.

    @ned-deily
    Copy link
    Member

    As of Xcode 13.3, released 2022-03-14 to support macOS 12.3 et al, the included Apple Clang (Apple clang version 13.1.6 (clang-1316.0.21.2)) now supports the --print-multiarch option and so Python 3.8 and 3.7 are now vulnerable to this issue when building with that version of Xcode or Apple Command Line Tools on macOS. While both 3.8 and 3.7 are now in the security-fix-only phase of their life cycles and are not fully supported on macOS 12, you should still be able to run ./configure without error using the current Apple Developer Tools. PR 31889 and PR 31890 are backports of this to 3.8 and 3.7. Setting to "release blocker" priority for upcoming 3.8 and 3.7 releases.

    @ned-deily ned-deily added 3.7 (EOL) end of life 3.8 only security fixes release-blocker and removed 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Mar 15, 2022
    @ned-deily ned-deily reopened this Mar 15, 2022
    @ned-deily ned-deily added 3.8 only security fixes release-blocker and removed 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes labels Mar 15, 2022
    @ned-deily ned-deily reopened this Mar 15, 2022
    @ned-deily
    Copy link
    Member

    New changeset 720bb45 by Ned Deily in branch '3.7':
    bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31890)
    720bb45

    @ambv
    Copy link
    Contributor

    ambv commented Mar 15, 2022

    New changeset dea270a by Ned Deily in branch '3.8':
    bpo-45405: Prevent internal configure error when running configure with recent versions of clang. (GH-28845) (GH-31889)
    dea270a

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @jessecureton jessecureton mentioned this issue Apr 12, 2022
    Merged
    @mansourmoufid mansourmoufid mentioned this issue Aug 25, 2022
    Closed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes build The build process and cross-build
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @ambv @ned-deily @miss-islington