This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: explore hashlib use of the Apple CryptoKit macOS
Type: enhancement Stage:
Components: Extension Modules, macOS Versions:
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, gregory.p.smith, ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2022-03-26 00:48 by gregory.p.smith, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg416032 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-26 00:48 in macOS 10.15+

This is a common place for platform specific hardware acceleration to be exposed to the user (especially on SoCs which often have non-standard hardware - Like Apples... which is presumably why they create this).

What they offer is limited, but when present and running on a recent enough macOS, using their and SHA2 and HMAC(SHA2) implementations as well as Insecure.SHA1 is probably better than OpenSSL's.  **Verify this.** It'd also allow those to be fast in a non-openssl build (as if anyone does those).

I know little about mac building and packaging and how to have something target an older OS and use a 10.15+ API. So if this winds up only being used from aarch64 macOS builds (10.15+ by definition IIRC?) that could also work.

I leave this issue for a macOS Apple API friendly person to take on.

This issue is cousin to the Linux one:
msg416111 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2022-03-27 08:35
A "problem" with CryptoKit is that it is a swift-only framework, which makes using those APIs harder from C code (not impossible).

The older Security framework also contains crypto APIs, but seems to have less support for modern algorithms (e.g. no support for Curve25519). 

TBH I'm not sure if it is worthwhile to look into this in CPython, or that we should rely on OpenSSL for any integration (similar to Christian Heimes opinion on using the system keystore in the ssl module).
msg416131 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2022-03-27 17:57
I only pointed to that API after a brief search without looking at details (Swift? oops!). If there is one available from C that'd also make sense to consider.

The only things I expect, relevant to hashlib, that would be accelerated by OS native APIs most platforms are SHA2, maybe SHA1, and sometimes HMAC using those.

I'm in no position to judge if there is value in using them, I'm just assuming there might be.  The irony is that builds without OpenSSL are rare, so unless the OS native APIs provide tangible benefits it may not matter.

(ex: the Linux APIs may allow for an efficient zero-copy variant of the new `hashlib.file_digest()` function)
msg416165 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2022-03-28 12:56
SecDigestTransformCreate() is probably a relevant API to look into, this seems to be supported from 10.7 until now.

A major disadvantage for us of this API is that it is a CoreFoundation API and because of that is problematic in pre-forking scenario's (that is, call in a child proces that's the result of fork-without-exec) because most if not all CoreFoundation types are not safe to use in these scenario's.

Apple also has an older crypto API, but that has been deprecated for a long time and should not be used.
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91280
2022-03-28 14:01:03corona10setnosy: + corona10
2022-03-28 12:56:19ronaldoussorensetmessages: + msg416165
2022-03-27 17:57:12gregory.p.smithsetmessages: + msg416131
2022-03-27 08:35:50ronaldoussorensetmessages: + msg416111
2022-03-26 00:48:30gregory.p.smithcreate