This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: _ssl doesn't build on OSX 10.11
Type: behavior Stage: resolved
Components: macOS Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: cjrh, matrixise, ned.deily, r.david.murray, ronaldoussoren, tdsmith, vstinner, willingc, zach.ware
Priority: normal Keywords:

Created on 2015-11-06 21:29 by matrixise, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (10)
msg254222 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2015-11-06 21:29
On OSX 10.11, if you don't use Homebrew or MacPorts, the _ssl module is not built with Makefile.

Homebrew supports OpenSSL and LibreSSL, but OpenSSL is in /usr/local/opt/openssl and not in /usr/local/ssl (see setup.py)

So, the next point, I am not sure, but the headers of OpenSSL has not provided with OSX 10.11
msg254287 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2015-11-07 17:19
If I read the issue correctly the OSX 10.11 SDK doesn't include OpenSSL headers. 

There's not a lot we can do about that beyond documenting this in the build instructions. 

It might be interesting to look into using Apple's crypto APIs instead of OpenSSL but that is a significant amount of work with limited pay-off (and a risk of introducing subtle differences between the OS X port and other ports)
msg254318 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-08 01:39
My understanding is that when we build an osx release, we bundle openssl.  So I think the appropriate solution here from our point of view is to include a pointer in the devguide about downloading and installing openssl-with-headers however it is we do that for a release build.
msg254321 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-11-08 03:52
"My understanding is that when we build an osx release, we bundle openssl."

Well, no, we don't exactly do that today.  We have recently been doing that for the 10.5 installer because the 10.5 system version of OpenSSL is so old as to be unusable for PyPI downloads (and pretty much everything else).  But the much more commonly used 10.6+ installer dynamically links with the latest (but deprecated) system OpenSSL libs shipped with OS X, libs that are old but usable and still receiving Apple security fixes on current systems.  The main reason for doing that is that this solution uses the Apple-provided system and user keychains for certificate management.  There are other drawbacks, though, and we have an open issue to deal with that.  (The python.org Pythons are not affected by the lack of headers since the libs are still shipped in OS X 10.11.)  What I think this issue should address is making it easier to build Python on 10.11 (and earlier systems) with newer versions of OpenSSL.  It should be easier to use OpenSSL libs supplied by popular third-party distributors like Homebrew and MacPorts.  Without having thought through all the details, I'm thinking it might be best to provide a configure-time check and option, possibly with standard options for the system,  Homebrew (default path), MacPorts (default path), and user-built SSL headers/libs.  It would also be good to provide something which will build the handful of important missing and/or newer third-party libs needed on OS X by the interpreter and standard library, like SSL libs, liblzma, and Tk.  But that's a separate issue.
msg255342 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-25 13:24
> Homebrew supports OpenSSL and LibreSSL, but OpenSSL is in /usr/local/opt/openssl and not in /usr/local/ssl (see setup.py)

Does it make sense to add a configure option to specify the path to OpenSSL?

> So, the next point, I am not sure, but the headers of OpenSSL has not provided with OSX 10.11

Is it possible or not to get headers of the OpenSSL system library?
msg255364 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2015-11-25 15:30
Apple no longer ships headers for OpenSSL in OSX SDK as shipped with Xcode 7.1, and I'd be surprised if that changes (without having any inside knowledge about this).

W.r.t. adding configure option for OpenSSL: That would be fine although that makes the detection of openssl even more convoluted that it currently is: there's also a search list in setup.py.   In the long run its probably better to move all library detection code to configure. See also Issue8406, Issue7713 and possibly more.
msg268636 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-06-15 23:33
Note that the Python Developer's Guide now contains instructions for how to build on OS X with third-party versions of OpenSSL:

https://docs.python.org/devguide/setup.html#build-dependencies
msg269102 - (view) Author: Carol Willing (willingc) * (Python committer) Date: 2016-06-23 04:48
After Ned's last message, perhaps it would be best to close this issue.
msg269111 - (view) Author: Caleb Hattingh (cjrh) * Date: 2016-06-23 06:00
I struggled with this issue, and eventually found the recommendations about linking with homebrew's OpenSSL on StackOverflow or similar, and then only later found this issue here (and with it the link to the devguide); but the *first* places I looked were the README in the source root, and then the README in the Mac/ directory. That may however just be ignorance on my part of where I should have been looking. Yet another reminder that I need to become much more familiar with the devguide.

The README only mentions the devguide in the context of contributing, but not that it will contain further information required for building. Under "Build Instructions", the README says:

***

Build Instructions
------------------

On Unix, Linux, BSD, OSX, and Cygwin:

    ./configure
    make
    make test
    sudo make install

This will install Python as python3.

You can pass many options to the configure script; run "./configure --help" to find out more.  On OSX and Cygwin, the executable is called python.exe;
elsewhere it's just python.

On Mac OS X, if you have configured Python with --enable-framework, you should use "make frameworkinstall" to do the installation.  Note that this installs the Python executable in a place that is not normally on your PATH, you may want to set up a symlink in /usr/local/bin.

***

It might be helpful to add to the README (in the "Build Instructions" section): 

"The devguide may include further information about specific build dependencies for your platform here: https://docs.python.org/devguide/setup.html#build-dependencies"
msg270169 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2016-07-11 09:50
The new paragraph about Homebrew for OSX is useful, we can close this issue.
History
Date User Action Args
2022-04-11 14:58:23adminsetgithub: 69758
2016-07-11 10:24:04berker.peksagsettype: behavior
stage: needs patch -> resolved
resolution: out of date
versions: - Python 3.4
2016-07-11 09:50:01matrixisesetstatus: open -> closed

messages: + msg270169
2016-06-23 06:00:46cjrhsetmessages: + msg269111
2016-06-23 04:48:20willingcsetnosy: + willingc
messages: + msg269102
2016-06-15 23:33:32ned.deilysetmessages: + msg268636
2016-06-15 23:29:58cjrhsetnosy: + cjrh
2015-11-25 15:30:48ronaldoussorensetmessages: + msg255364
2015-11-25 13:24:11vstinnersetnosy: + vstinner
messages: + msg255342
2015-11-08 03:52:14ned.deilysetmessages: + msg254321
2015-11-08 01:39:04r.david.murraysetnosy: + r.david.murray
messages: + msg254318
2015-11-07 17:19:35ronaldoussorensetmessages: + msg254287
title: _ssl doesn't build on OS X 10.11 without third-party ssl headers -> _ssl doesn't build on OSX 10.11
2015-11-06 22:03:43ned.deilysetassignee: ned.deily
title: _ssl doesn't build on OSX 10.11 -> _ssl doesn't build on OS X 10.11 without third-party ssl headers
2015-11-06 21:53:38tdsmithsetnosy: + tdsmith
2015-11-06 21:31:46zach.waresetnosy: + ronaldoussoren, ned.deily, zach.ware
stage: needs patch

components: + macOS
versions: + Python 2.7, Python 3.4, Python 3.5, Python 3.6
2015-11-06 21:29:59matrixisecreate