This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: python3.9-intel64 hardened runtime not enabled
Type: compile error Stage: resolved
Components: macOS Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: bettyrab, ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2021-08-17 09:18 by bettyrab, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (9)
msg399721 - (view) Author: tester (bettyrab) Date: 2021-08-17 09:18
When trying too build the python framework using the method below and you try to get it notarized you get the following error.

"path": "munkitools-5.5.0.4362.pkg/munkitools_python.pkg Contents/Payload/usr/local/munki/Python.framework/Versions/3.9/bin/python3.9-intel64",
"message": "The executable does not have the hardened runtime enabled.",

https://github.com/munki/munki/blob/main/code/tools/build_python_framework.sh


The package get built using this ttps://github.com/lifeunexpected/Scripts

This issue happens on python 3.9.5 and 3.9.6 earlier versions did not include python3.9-intel64.
msg399725 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2021-08-17 09:41
I'm not sure what your question is. But to enable the hardened runtime you need to sign the binaries using ``--options runtime`` as one of the options.
msg399736 - (view) Author: tester (bettyrab) Date: 2021-08-17 10:34
This part of the script is used to notarize the python framework and it worked with python 3.9.4 and earlier but with 3.9.5 and 3.9.6 it stopped working because of "/python3.9-intel64",
"message": "The executable does not have the hardened runtime enabled.

#get current python version used in Munki build so that it doesn't have to be hardcoded
PYTHON_FRAMEWORK_VERSION=$(ls Python.framework/Versions | grep -v "Current")

find $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/lib/ -type f -perm -u=x -exec codesign --force --deep --verbose -s "$DevApp" {} \;
find $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/bin/ -type f -perm -u=x -exec codesign --force --deep --verbose -s "$DevApp" {} \;

find $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/lib/ -type f -name "*dylib" -exec codesign --force --deep --verbose -s "$DevApp" {} \;
find $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/lib/ -type f -name "*so" -exec codesign --force --deep --verbose -s "$DevApp" {} \;

/usr/libexec/PlistBuddy -c "Add :com.apple.security.cs.allow-unsigned-executable-memory bool true" $MUNKIROOT/entitlements.plist

codesign --force --options runtime --entitlements $MUNKIROOT/entitlements.plist --deep --verbose -s "$DevApp" $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/Resources/Python.app/

codesign --force --options runtime --entitlements $MUNKIROOT/entitlements.plist --deep --verbose -s "$DevApp" $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/bin/"python$PYTHON_FRAMEWORK_VERSION"
codesign --force --deep --verbose -s  "$DevApp" $MUNKIROOT/Python.framework
msg399738 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2021-08-17 10:43
I don't know what creates "python3.9-intel64", but this is not created by the regular Python build process.  It might be created by one of the build scripts you use.
msg399742 - (view) Author: tester (bettyrab) Date: 2021-08-17 11:22
oh i didn't now that so thank you so much i will look into that maybe thats where my problem is :)
msg399821 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-08-18 04:32
The "python3.x-intel64" executable was added for macOS "universal2" builds by Issue44009 in the 3.9.5 and 3.8.10 releases. As described in the issue and the changelog entries, the rationale is:

"Provide “python3.x-intel64” executable to allow reliably forcing macOS universal2 framework builds to run under Rosetta 2 Intel-64 emulation on Apple Silicon Macs. This can be useful for testing or when universal2 wheels are not yet available."

We provided an analogous "python3.x-32" executable for macOS "intel" (x64_64 and i386 archs) builds for a similar reason.

Although it is mentioned in the changelog (and the python.org macOS installer ReadMe), I didn't anticipate that there would be anyone attempting to build and submit a universal binary for Apple notarization and who would be affected by this change: sorry about that! You can just either add the new executable to your codesigning script or you could delete the file before codesigning if you don't have a need to provide it to your users: it isn't needed unless you want to reliably force running under Rosetta 2 Intel emulation on an Apple Silcon Mac.
msg399930 - (view) Author: tester (bettyrab) Date: 2021-08-19 19:20
I found the fix for it earlier today and of course the fix is always right in front of you, you just need to look in the right place lol jokes on me.

I orginnaly had this and a few other things in my notarization script to get python notarized.
codesign --force --options runtime --entitlements $MUNKIROOT/entitlements.plist --deep --verbose -s "$DevApp" $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/bin/"python$PYTHON_FRAMEWORK_VERSION"

So i just had to add the stuff below to make python3.x-intel64 also get notarized properly

codesign --force --options runtime --entitlements $MUNKIROOT/entitlements.plist --deep --verbose -s "$DevApp" $MUNKIROOT/Python.framework/Versions/$PYTHON_FRAMEWORK_VERSION/bin/"python$PYTHON_FRAMEWORK_VERSION-intel64"

https://github.com/lifeunexpected/Scripts/blob/master/Munki.Notarize.zsh

I mostly posted this in case anyone else randomly googles this type off issue so maybe it can help them.
msg400368 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2021-08-26 18:48
@ned: the "reliable way to run under rosetta" is using "arch -x86_64 python3". I don't particularly like having another executable to accomplish the same goal.
msg402685 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-09-26 22:26
> the "reliable way to run under rosetta" is using "arch -x86_64 python3". I don't particularly like having another executable to accomplish the same goal.

Unfortunately, using "arch" is *not* a reliable way in general. It works fine for the initial invocation of the interpreter but things fall apart when a Python program tries to invoke another Python interpreter in a subprocess. The standard idiom for finding the location of the running interpreter is to use the contents of sys.executable. But that merely contains the name of the interpreter file, thus the additional "arch" arguments are lost and the interpreter will launch in the subprocess using the default arch, which may result in a failure (if you are lucky) or, worse, will silently run under the wrong architecture potentially giving misleading results. The most notable example of this is while running the python test suite. We ran into this same issue years ago with the "intel" universal-arch build option which is why we added a "python3-32" executable to provide a reliable way to force 32-bit execution on a 64-bit-capable Mac. The need for "python3-intel64" will eventually go away once Rosetta2 is no longer supported.
History
Date User Action Args
2022-04-11 14:59:48adminsetgithub: 89096
2021-09-26 22:26:46ned.deilysetmessages: + msg402685
2021-08-26 18:48:39ronaldoussorensetmessages: + msg400368
2021-08-19 19:20:41bettyrabsetmessages: + msg399930
2021-08-18 04:32:33ned.deilysetstatus: open -> closed
resolution: not a bug
messages: + msg399821

stage: resolved
2021-08-17 11:22:44bettyrabsetmessages: + msg399742
2021-08-17 10:43:39ronaldoussorensetmessages: + msg399738
2021-08-17 10:34:36bettyrabsetmessages: + msg399736
2021-08-17 09:41:49ronaldoussorensetmessages: + msg399725
2021-08-17 09:18:56bettyrabcreate