Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate using Apple clang for building OS X installers #63219

Closed
ned-deily opened this issue Sep 14, 2013 · 14 comments
Closed

Investigate using Apple clang for building OS X installers #63219

ned-deily opened this issue Sep 14, 2013 · 14 comments
Assignees
Labels
3.7 (EOL) end of life build The build process and cross-build OS-mac

Comments

@ned-deily
Copy link
Member

BPO 19019
Nosy @smontanaro, @rhettinger, @ronaldoussoren, @ned-deily, @bitdancer
PRs
  • [2.7] Backport macOS universal build and installer fixes from 3.6. #6469
  • [2.7] Fix errant NEWS item: bpo-19019 -> bpo-17128 #6470
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ned-deily'
    closed_at = <Date 2018-04-15.20:41:24.876>
    created_at = <Date 2013-09-14.20:31:13.783>
    labels = ['OS-mac', 'build', '3.7']
    title = 'Investigate using Apple clang for building OS X installers'
    updated_at = <Date 2018-04-15.20:41:24.872>
    user = 'https://github.com/ned-deily'

    bugs.python.org fields:

    activity = <Date 2018-04-15.20:41:24.872>
    actor = 'ned.deily'
    assignee = 'ned.deily'
    closed = True
    closed_date = <Date 2018-04-15.20:41:24.876>
    closer = 'ned.deily'
    components = ['Build', 'macOS']
    creation = <Date 2013-09-14.20:31:13.783>
    creator = 'ned.deily'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 19019
    keywords = ['patch']
    message_count = 14.0
    messages = ['197729', '198028', '198029', '198030', '198031', '198032', '198176', '200367', '201198', '201199', '201220', '315293', '315295', '315340']
    nosy_count = 7.0
    nosy_names = ['skip.montanaro', 'rhettinger', 'ronaldoussoren', 'ned.deily', 'r.david.murray', 'python-dev', 'abarnert']
    pr_nums = ['6469', '6470']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue19019'
    versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

    @ned-deily
    Copy link
    Member Author

    On the python-dev mailing list, Raymond Hettinger asks:

    I was exercising the alpha two release of 3.4 and noticed that
    it is still being built under GCC 4.2.1.

    Is there any reason we have to use an old compiler?

    Yes, kinda. It's because the 64-bit/32-bit installer we supply is designed to run on multiple versions of OS X: 10.6 (Snow Leopard), 10.7, 10.8, and the upcoming 10.9. The safest way to ensure that the bits produced (executables, shared libs, and .so bundles) are linked with compatible versions of system shared libraries and frameworks is to build on the lowest supported system, 10.6, or - carefully - with a 10.6 SDK on a newer system. The standard compiler in the Apple Developer Tools for 10.6 is gcc 4.2. The most recent Developer Tools for 10.8 and 10.7 systems, Xcode 4.6.x, have a mature clang but do not provide a 10.6 SDK. Even with using an SDK, it's still possible to end up inadvertently linking with the wrong versions of system libraries. We have been burned by that in the past.

    I would like to see it built under the latest version of Clang
    (like the other tools on the Mac) or under GCC 4.8.1.
    I've better using the latter for many months. It works great
    and produces *much* better code (in terms of readability
    for those of us who look at assembly output and in terms
    of speed (approx 20-25% better for cpu bound code)).

    I'm wondering whether is some issue that forces us to use
    GCC 4.2.1 or whether that is just an artifact of continuing
    to do what we've always been doing.

    I'd like to move to Apple's latest clang, as well. (Apple is dropping gcc totally from the next release of Xcode.) There are a couple of approaches that may work and still reliably support multiple versions of OS X. Investigating this is on my list of things to do prior to the next alpha.

    @ned-deily ned-deily self-assigned this Sep 14, 2013
    @ned-deily ned-deily added build The build process and cross-build OS-mac labels Sep 14, 2013
    @smontanaro
    Copy link
    Contributor

    If you find it difficult to find people with 10.6 systems to build a Python dmg for that platform, give me a shout. I'm not planning on replacing my system anytime soon. I have 10.6.8 installed, and what I believe to be the last Xcode version to support it, 3.2.something.

    @ned-deily
    Copy link
    Member Author

    Thanks, Skip. At the moment, finding platforms for building and testing is not a problem: I have at hand running versions of the latest releases for 10.3 onwards. :) (Buildbots for automated testing are another issue, though. We have a critical need for buildbots running more recent versions of OS X, though. At the moment, the only OS X buildbot is one running 10.4 (Tiger). If anyone is able to provide newer ones, please see http://python.org/dev/buildbot/. Or, if someone would like to donate hardware, I may be able to arrange hosting for it.)

    @smontanaro
    Copy link
    Contributor

    I guess snakebite (http://www.snakebite.net/) never materialized?

    @ned-deily
    Copy link
    Member Author

    Snakebite did provide some additional OS X buildbots for a while but, AFAIK, they have been offline for some months now.

    @bitdancer
    Copy link
    Member

    Snakebite is live, and python committers can get shell access on the bots, as originally planned. Some of the bots are offline, but most are running. I don't see any OS X servers in the list, though, unless Antoine deleted them (but there are several other currently-offline ones he hasn't deleted, so I doubt that).

    @bitdancer
    Copy link
    Member

    I've spun up a 10.6.8 OS X buildbot.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 19, 2013

    New changeset 5e385dcfbd32 by Ned Deily in branch 'default':
    Issue bpo-19019: Change the OS X installer build script to use CFLAGS instead
    http://hg.python.org/cpython/rev/5e385dcfbd32

    @abarnert
    Copy link
    Mannequin

    abarnert mannequin commented Oct 25, 2013

    Now that Xcode 5 is out, and does not have anything named "gcc-4.2" (it _does_ still have something called "gcc", which is now a wrapper around clang instead of a wrapper around llvm-gcc-4.2), this seems a lot more critical, because (as far as I can tell) nobody with Xcode 5 can build C extensions for python.org Python installations.

    And that also implies that any changes need to be backported to 2.7 and 3.3, unless the plan is that only Python 3.4+ should work with Xcode 5.

    @ned-deily
    Copy link
    Member Author

    "nobody with Xcode 5 can build C extensions for python.org Python installations"

    Andrew, can you elaborate on what problems you or others are seeing? Xcode 5 should not be an issue for users of the current python.org Pythons. When building extension modules, if the compiler Python was built with is not available Distutils in the current 2.7.x, 3.3.x, and 3.4.x release will substitute clang. This assumes they have the Command Line Tools installed, either through Xcode or the standalone CLT package and assumes that the user is not overriding defaults by setting environment variables like CC and LDSHARED which Distutils will honor without attempting to modify them. This should all be independent of what compiler Python was built with.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Oct 25, 2013

    New changeset 323f1dcceb73 by Ned Deily in branch '2.7':
    Issue bpo-19019: Change the OS X installer build script to use CFLAGS instead
    http://hg.python.org/cpython/rev/323f1dcceb73

    New changeset 689b5c2d6959 by Ned Deily in branch '3.3':
    Issue bpo-19019: Change the OS X installer build script to use CFLAGS instead
    http://hg.python.org/cpython/rev/689b5c2d6959

    @ned-deily
    Copy link
    Member Author

    New changeset ee8e4b6 by Ned Deily in branch '2.7':
    [2.7] Backport macOS universal build and installer fixes from 3.6. (GH-6469)
    ee8e4b6

    @ned-deily
    Copy link
    Member Author

    New changeset 93c91ac by Ned Deily in branch '2.7':
    [2.7] Fix errant NEWS item: bpo-19019 -> bpo-17128 (GH-6470)
    93c91ac

    @ned-deily
    Copy link
    Member Author

    As of 3.6.5, 2.7.15, and the upcoming 3.7.0. the macOS installers now build with a generic 'gcc' specified, rather than the somewhat confusing 'gcc-4.2' and we also provide a new 10.9+ 64-bit-only installer variant that is built with clang. So I think we can finally close this issue. Thanks all!

    @ned-deily ned-deily added the 3.7 (EOL) end of life label Apr 15, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life build The build process and cross-build OS-mac
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants