classification
Title: Remove bundled libffi for OSX
Type: enhancement Stage: resolved
Components: Build, ctypes, macOS Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder: Support macOS 11 and Apple Silicon Macs
View: 41100
Assigned To: Nosy List: koobs, matrixise, ned.deily, r.david.murray, ronaldoussoren, yan12125, zach.ware
Priority: normal Keywords: patch

Created on 2016-10-20 18:40 by zach.ware, last changed 2020-11-08 12:07 by ronaldoussoren. This issue is now closed.

Files
File name Uploaded Description Edit
remove_libffi_osx.diff zach.ware, 2016-10-20 18:40 review
Messages (10)
msg279063 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-10-20 18:40
Here's a patch that at least allows _ctypes to be built against a Homebrew libffi and pass the test_ctypes.  It is almost certainly not a complete patch, but may serve as a basis for something that will actually work.
msg279064 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2016-10-20 18:41
I will test on my OSX (El Capitan)
msg279065 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-10-20 18:51
With this change pkg-config becomes mandatory on macOS, too. As Ned has mentioned in issue28207, pkg-config is not pre-installed with macOS.
msg279067 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-10-20 19:23
Yes, we shouldn't be depending on pkg-config being available.  I am not at all keen on adding a dependency on a third-party library supplied by another distributor. What I would like to see is: (1) add libffi to the third-party libs built and used for the macOS installer build, which also means on all supported versions; (2) then, more generally, refactor the third-party lib builds (e.g. OpenSSL, SQLite, xz, ncurses, et al, and then libffi) out of the installer build script and provide an option to ./configure to allow the third-party libs to be built and used in a regular developer build as the absence of or lack of updates to them in macOS releases is a growing problem.  At that point we can get rid of the bundled libffi source.  At the moment, AFAIK, the presence of the bundled libffi source is not causing any problems so this isn't a critical problem, unlike the case with OpenSSL.
msg279069 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2016-10-20 19:35
On my OSX with El Capitan and HomeBrew

Configuration:

env PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig ./configure --prefix=$PWD-build

Compilation:

make

Tests:

./python.exe -c 'import ctypes'

Results:
* No SIGSEGV
* Add a new dependency to pkg-config :/
msg279122 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-10-21 11:25
Does the upstream libffi work reliably on OSX by this time?  

The bundled version of libffi was extracted from PyObjC, and that copy is itself a patched version of the system libffi on OSX. The reason I don't use the system libffi is both that I don't want to rely on older version of libffi, and because the system libffi causes crashes in PyObjC's testsuite (which contains a lot of edge cases). 

I haven't looked into using the upstream version of libffi yet. PyObjC's version of libffi is a fairly old fork of upstream libffi where support for other systems was removed and support for darwin/x86 (and later darwin/x86-64) as added. The former at a time that the upstream libffi didn't support darwin/x86 at all (and at a time that darwin/x86 itself wasn't available on consumer devices).
msg279123 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2016-10-21 11:30
Hi Ronald,

As you can see in my message #msg279069 I have tested with the last version of libffi (installed with HomeBrew), so in this case, I think we can say "Yes" to your question, the current release of libffi works fine with OSX. Now, I can check more deeply and give you a feedback.
msg289707 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2017-03-16 05:48
Update: my pull request at https://github.com/libffi/libffi/pull/288 is merged. Future libffi releases don't need pkg-config anymore. However, I guess the system copy won't be updated in near future.
msg379419 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-23 09:08
added 41100 as a dependency because that switches to the system libffi (as a side effect of larger changes)
msg380548 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-11-08 12:07
This was implemented in bpo-41100
History
Date User Action Args
2020-11-08 12:07:24ronaldoussorensetstatus: open -> closed
superseder: Support macOS 11 and Apple Silicon Macs
messages: + msg380548

dependencies: - Support macOS 11 and Apple Silicon Macs
resolution: fixed
stage: test needed -> resolved
2020-10-23 09:08:35ronaldoussorensetdependencies: + Support macOS 11 and Apple Silicon Macs
messages: + msg379419
2017-03-16 05:48:55yan12125setmessages: + msg289707
2016-10-21 11:35:59koobssetnosy: + koobs
2016-10-21 11:30:28matrixisesetmessages: + msg279123
2016-10-21 11:25:21ronaldoussorensetmessages: + msg279122
2016-10-20 19:35:30matrixisesetmessages: + msg279069
2016-10-20 19:23:16ned.deilysetmessages: + msg279067
2016-10-20 18:51:02yan12125setmessages: + msg279065
2016-10-20 18:41:44matrixisesetmessages: + msg279064
2016-10-20 18:40:27zach.warecreate