classification
Title: Remove bundled libffi for OSX
Type: enhancement Stage: test needed
Components: Build, ctypes, macOS Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Chi Hsuan Yen, koobs, matrixise, ned.deily, r.david.murray, ronaldoussoren, zach.ware
Priority: normal Keywords: patch

Created on 2016-10-20 18:40 by zach.ware, last changed 2017-03-16 05:48 by Chi Hsuan Yen.

Files
File name Uploaded Description Edit
remove_libffi_osx.diff zach.ware, 2016-10-20 18:40 review
Messages (8)
msg279063 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-10-20 18:40
Here's a patch that at least allows _ctypes to be built against a Homebrew libffi and pass the test_ctypes.  It is almost certainly not a complete patch, but may serve as a basis for something that will actually work.
msg279064 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2016-10-20 18:41
I will test on my OSX (El Capitan)
msg279065 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2016-10-20 18:51
With this change pkg-config becomes mandatory on macOS, too. As Ned has mentioned in issue28207, pkg-config is not pre-installed with macOS.
msg279067 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-10-20 19:23
Yes, we shouldn't be depending on pkg-config being available.  I am not at all keen on adding a dependency on a third-party library supplied by another distributor. What I would like to see is: (1) add libffi to the third-party libs built and used for the macOS installer build, which also means on all supported versions; (2) then, more generally, refactor the third-party lib builds (e.g. OpenSSL, SQLite, xz, ncurses, et al, and then libffi) out of the installer build script and provide an option to ./configure to allow the third-party libs to be built and used in a regular developer build as the absence of or lack of updates to them in macOS releases is a growing problem.  At that point we can get rid of the bundled libffi source.  At the moment, AFAIK, the presence of the bundled libffi source is not causing any problems so this isn't a critical problem, unlike the case with OpenSSL.
msg279069 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2016-10-20 19:35
On my OSX with El Capitan and HomeBrew

Configuration:

env PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig ./configure --prefix=$PWD-build

Compilation:

make

Tests:

./python.exe -c 'import ctypes'

Results:
* No SIGSEGV
* Add a new dependency to pkg-config :/
msg279122 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-10-21 11:25
Does the upstream libffi work reliably on OSX by this time?  

The bundled version of libffi was extracted from PyObjC, and that copy is itself a patched version of the system libffi on OSX. The reason I don't use the system libffi is both that I don't want to rely on older version of libffi, and because the system libffi causes crashes in PyObjC's testsuite (which contains a lot of edge cases). 

I haven't looked into using the upstream version of libffi yet. PyObjC's version of libffi is a fairly old fork of upstream libffi where support for other systems was removed and support for darwin/x86 (and later darwin/x86-64) as added. The former at a time that the upstream libffi didn't support darwin/x86 at all (and at a time that darwin/x86 itself wasn't available on consumer devices).
msg279123 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2016-10-21 11:30
Hi Ronald,

As you can see in my message #msg279069 I have tested with the last version of libffi (installed with HomeBrew), so in this case, I think we can say "Yes" to your question, the current release of libffi works fine with OSX. Now, I can check more deeply and give you a feedback.
msg289707 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2017-03-16 05:48
Update: my pull request at https://github.com/libffi/libffi/pull/288 is merged. Future libffi releases don't need pkg-config anymore. However, I guess the system copy won't be updated in near future.
History
Date User Action Args
2017-03-16 05:48:55Chi Hsuan Yensetmessages: + msg289707
2016-10-21 11:35:59koobssetnosy: + koobs
2016-10-21 11:30:28matrixisesetmessages: + msg279123
2016-10-21 11:25:21ronaldoussorensetmessages: + msg279122
2016-10-20 19:35:30matrixisesetmessages: + msg279069
2016-10-20 19:23:16ned.deilysetmessages: + msg279067
2016-10-20 18:51:02Chi Hsuan Yensetmessages: + msg279065
2016-10-20 18:41:44matrixisesetmessages: + msg279064
2016-10-20 18:40:27zach.warecreate