Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_tkinter built on macOS 10.14 does not link to Tcl and Tk in /Library/Frameworks #79137

Closed
wordtech mannequin opened this issue Oct 11, 2018 · 9 comments
Closed

_tkinter built on macOS 10.14 does not link to Tcl and Tk in /Library/Frameworks #79137

wordtech mannequin opened this issue Oct 11, 2018 · 9 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes OS-mac

Comments

@wordtech
Copy link
Mannequin

wordtech mannequin commented Oct 11, 2018

BPO 34956
Nosy @ronaldoussoren, @ned-deily, @tonybaloney, @miss-islington, @dimpase
PRs
  • bpo-34956: Fix macOS _tkinter use of Tcl/Tk in /Library/Frameworks #20171
  • bpo-34956: edit and format NEWS item in 3.9.0b1 changelog #20255
  • [3.9] bpo-34956: edit and format better NEWS item in 3.9.0b1 changelog (GH-20255) #20257
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ned-deily'
    closed_at = None
    created_at = <Date 2018-10-11.03:29:07.952>
    labels = ['OS-mac', '3.7', '3.8']
    title = '_tkinter built on macOS 10.14 does not link to Tcl and Tk in /Library/Frameworks'
    updated_at = <Date 2021-11-04.14:23:12.227>
    user = 'https://bugs.python.org/wordtech'

    bugs.python.org fields:

    activity = <Date 2021-11-04.14:23:12.227>
    actor = 'erlendaasland'
    assignee = 'ned.deily'
    closed = False
    closed_date = None
    closer = None
    components = ['macOS']
    creation = <Date 2018-10-11.03:29:07.952>
    creator = 'wordtech'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 34956
    keywords = ['patch']
    message_count = 8.0
    messages = ['327508', '327522', '327524', '327556', '337562', '369195', '369434', '369435']
    nosy_count = 6.0
    nosy_names = ['ronaldoussoren', 'wordtech', 'ned.deily', 'anthonypjshaw', 'miss-islington', 'dimpase']
    pr_nums = ['20171', '20255', '20257']
    priority = 'high'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue34956'
    versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

    @wordtech
    Copy link
    Mannequin Author

    wordtech mannequin commented Oct 11, 2018

    I'm trying to build Python 3.7.0 on macOS 10.14, and Tkinter is not linking to my installation of Tcl/Tk 8.6.8 in /Library/Frameworks. Instead it is linking to the ancient 8.5 Tk installed in /System/Library/Frameworks. My usual way of forcing Python to link to my installation is to edit setup.py and comment out all search directories except /Library/Frameworks, but that seems to be ignored here. My basic invocation is "./configure --enable-framework" which, along with omitting the system libraries from setup.py, has always been sufficient in the past. Please advise.

    @wordtech wordtech mannequin added OS-mac 3.7 (EOL) end of life labels Oct 11, 2018
    @ned-deily
    Copy link
    Member

    macOS 10.14 appears to have introduced some subtle differences in the search order for finding header files and/or frameworks and libraries. I'm not 100% sure I understand completely what has changed but I'm confident that the workaround outlined below should get you going until we have a better solution (but beware of the potential gotcha).

    Python has never fully supported building only from an Xcode-only installation, i.e. we have required at a minimum installing system header files into /usr/include et al by using "xcode-select --install". As of 10.14, xcode-select no longer installs header files in /usr/include. So that further cripples Python builds in that header files for several third-party libraries shipped with macOS are no longer found, like zlib and sqlite3. When using an Xcode-only installation (no Command Line Tools) in previous releases of macOS, I believe it was the case that essentially the system root directory ('/') was searched by the compiler tool chain for header files, for libraries, and for frameworks, and for frameworks the long standard fallback framework search path order was honored by default: first search /Library/Frameworks, then /System/Library/Frameworks. That worked fine if you were installing frameworks like Tcl and TK into /Library/Frameworks and wanting them to override the system ones. If you did not install the Command Line Tools, then the tool chain used the MacOSX.sdk embedded in Xcode.app as the system root for searching. By default, that's found in:

    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

    With an unmodified Xcode install, that directory contains usr and System directories but no Library. So the contents of /Library were ignored by the tool chain with the net effect that _tkinter builds would also link with the Apple-supplied Tcl and Tk 8.5 in /System/Library/Frameworks.

    With 10.14, Xcode-only installs still work the same way but now Command Line Tools seem to effectively use the SDK embedded in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk as the system root, rather than the system /. Just like the Xcode copy of the sdk, the CLT copy does not have a Library directory so now the results are the same as Xcode-only builds: _tkinter only links with the /System/Library Tcl and Tk. From the viewpoint of developers trying to build applications for distribution to others, it makes sense to try to ensure that /Library does not influence the build because /Library is under user control, not the system (Apple).

    Until we "recently" started shipping our private version of Tcl and Tk 8.6.x, which we do not install to /Library btw, it made sense to default to linking to /Library and falling back to /System/Library, allowing optional use of a third-party Tcl/Tk (like from ActiveState) and it all sort of worked if you used the tool chain from the CLT. But with 10.14, that no longer works. Python's build system complicates this all in large part because the top-level setup.py tries to guess what directories the compiler tool chain are going to search; that was always kind of iffy and now it's much more broken with macOS 10.14. There are ways to work around most of the issues but we need to have better default behavior. That's kind of a bigger deal.

    For the specific case of building _tkinter to link with Tcl and Tk frameworks in /Library, I think the easiest workaround is to add a Library symlink to the SDK in use, either under /Library/Developer/CommandTools or /Applications/Xcode.app. "xcode-select --print-path" will show which one is in use, but, alas, not the full path to the SDK.

    either:
    $ xcode-select --print-path
    /Library/Developer/CommandLineTools
    $ cd /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/

    or:
    $ xcode-select --print-path
    /Applications/Xcode.app/Contents/Developer
    $ xcodebuild -version -sdk macosx Path
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
    $ cd $(xcodebuild -version -sdk macosx Path)

    You can then carefully create a symlink from the SDK to /Library.

    $ sudo ln -s /Library .

    Now the Python builds should find the Current version of Tcl and Tk frameworks in /Library.

    *But* be aware that this might affect other builds if there are other frameworks you have installed in /Library. So this workaround should be used with caution and you might consider removing the symlink once you are done building Python.

    There are other, more complicated workarounds but this seems to me to be the simplest and most foolproof as long as one is aware of the risks and is prepared to potentially recreate the Library symlink after software updates to Xcode or the Command Line Tools.

    @ned-deily ned-deily added the 3.8 only security fixes label Oct 11, 2018
    @ned-deily ned-deily changed the title 3.7.0 _tkinter module links against /System/Library/Frameworks _tkinter built on macOS 10.14 does not link to Tcl and Tk in /Library/Frameworks Oct 11, 2018
    @ned-deily ned-deily self-assigned this Oct 11, 2018
    @ned-deily
    Copy link
    Member

    Correction: in the original second paragraph, the fourth sentence obviously has the case backwards. It should read:

    "When using a *Command Line Tools installation (no Xcode)* in previous releases of macOS, I believe it was the case that essentially the system root directory ('/') was searched by the compiler tool chain for header files, for libraries, and for frameworks, and for frameworks the long standard fallback framework search path order was honored by default: first search /Library/Frameworks, then /System/Library/Frameworks. [...]"

    @wordtech
    Copy link
    Mannequin Author

    wordtech mannequin commented Oct 12, 2018

    Thank you, this helped.

    @dimpase
    Copy link
    Mannequin

    dimpase mannequin commented Mar 9, 2019

    On https://bugs.python.org/issue36231 I propose a way to build without creating any symlinks.

    @ned-deily
    Copy link
    Member

    New changeset 1731d6d by Ned Deily in branch 'master':
    bpo-34956: Fix macOS _tkinter use of Tcl/Tk in /Library/Frameworks (GH-20171)
    1731d6d

    @ned-deily
    Copy link
    Member

    New changeset bac170c by Ned Deily in branch 'master':
    bpo-34956: edit and format better NEWS item in 3.9.0b1 changelog (GH-20255)
    bac170c

    @miss-islington
    Copy link
    Contributor

    New changeset e7bf868 by Miss Islington (bot) in branch '3.9':
    bpo-34956: edit and format better NEWS item in 3.9.0b1 changelog (GH-20255)
    e7bf868

    @ahmedsayeed1982 ahmedsayeed1982 mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) and removed OS-mac 3.8 only security fixes labels Nov 4, 2021
    @erlend-aasland erlend-aasland added OS-mac 3.8 only security fixes and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Nov 4, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @erlend-aasland
    Copy link
    Contributor

    Seems that this issue can be closed. Ned?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes OS-mac
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants