This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: `configure` script incorrectly detects symbols as available on Mac w/ Xcode 8+
Type: compile error Stage: resolved
Components: Build, macOS Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder: Support macOS 11 and Apple Silicon Macs
View: 41100
Assigned To: Nosy List: Maxime Belanger, ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2017-09-06 02:33 by Maxime Belanger, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg301432 - (view) Author: Maxime Belanger (Maxime Belanger) Date: 2017-09-06 02:33
At Dropbox, we use a custom fork of Python to deploy our Desktop Client onto various platforms, including macOS. We currently use the Mac OS X 10.11 SDK, but are considering updating to the macOS 10.12 SDK, which introduced new low-level functions, such as `getentropy`. Apple uses a series of "availability macros" with new warnings in Clang to consider the case when the `MACOSX_DEPLOYMENT_TARGET` is below the introduction version for a given symbol.

This has caused significant issues for `autoconf`-based projects, as running `configure` mistakes the built version into believing certain symbols are available, when they're not. As a result, deploying a viable build to users on older versions of Mac is rather complicated.

Current "solutions" (in recent versions of Xcode) include:

* Passing `-no_weak_imports` to the linker, which will fail any build using weakly-linked symbols (a pretty big hammer).
* A new compiler warning `-Wunguarded-availability`, which detects uses that contradict the `MACOSX_DEPLOYMENT_TARGET` using some AST tricks.

In our testing, neither had any effect on Python's `configure`, though these solutions worked for some other projects. We've implemented a workaround based on patching `pyconfig.h` after `configure` is complete, but ideally, Python's `configure` should detect `MACOSX_DEPLOYMENT_TARGET` and act accordingly (e.g. if `MACOSX_DEPLOYMENT_TARGET` is below 10.12, `getentropy` should not be considered available).

The following symbols are affected:

New in 10.13 (Python 3 only):
    utimensat       HAVE_UTIMENSAT
    futimens        HAVE_FUTIMENS

New in 10.12:
    getentropy      HAVE_GETENTROPY (Python 2/3)
    clock_gettime   HAVE_CLOCK_GETTIME (Python 2/3)
    clock_settime   HAVE_CLOCK_SETTIME (Python 3)
    clock_getres    HAVE_CLOCK_GETRES (Python 3)

New in 10.10 (Python 3 only):
    fstatat         HAVE_FSTATAT
    faccessat       HAVE_FACCESSAT
    fchmodat        HAVE_FCHMODAT
    fchownat        HAVE_FCHOWNAT
    linkat          HAVE_LINKAT
    fdopendir       HAVE_FDOPENDIR
    mkdirat         HAVE_MKDIRAT
    renameat        HAVE_RENAMEAT
    unlinkat        HAVE_UNLINKAT
    readlinkat      HAVE_READLINKAT
    symlinkat       HAVE_SYMLINKAT
    openat          HAVE_OPENAT
msg301451 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2017-09-06 07:15
An alternative solution, which is more invasive is to explicitly use weak linking, as is already used for some ancient APIs in posixmodule.c (look for "lchown" in Modules/posixmodule.c).

This is more invasive, but would be helpful for upstream as well because this allows building binaries that work on older versions of macOS but still make it possible to use new features when running on newer releases (and hence allows us to have a single installer for macOS 10.6+ without loosing features).
msg301499 - (view) Author: Maxime Belanger (Maxime Belanger) Date: 2017-09-06 18:50
That'd be even better :). I'll also note that this is slightly worse in Python 3 vs Python 2 as there are way more of these to deal with.
msg379421 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-23 09:16
Added 41100 as a dependency because the active PR for that issue will fix this issue as well.
msg380541 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-08 09:27
This is fixed in bpo-41100
Date User Action Args
2022-04-11 14:58:51adminsetgithub: 75540
2020-11-08 09:27:53ronaldoussorensetstatus: open -> closed
superseder: Support macOS 11 and Apple Silicon Macs
messages: + msg380541

dependencies: - Support macOS 11 and Apple Silicon Macs
resolution: fixed
stage: resolved
2020-10-23 09:16:08ronaldoussorensetdependencies: + Support macOS 11 and Apple Silicon Macs
messages: + msg379421
2017-09-06 18:50:59Maxime Belangersetmessages: + msg301499
2017-09-06 07:15:33ronaldoussorensetmessages: + msg301451
2017-09-06 02:34:35Maxime Belangersetnosy: + ned.deily, ronaldoussoren
type: compile error
components: + Build, macOS
2017-09-06 02:33:46Maxime Belangercreate