classification
Title: Compiling Python 3.6 from source on MacOS X Sierra
Type: Stage: resolved
Components: Build Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Tommy.Carstensen, doerwalter, gregcouch, ned.deily, yan12125
Priority: normal Keywords:

Created on 2016-12-28 20:02 by doerwalter, last changed 2017-10-05 21:11 by gregcouch. This issue is now closed.

Files
File name Uploaded Description Edit
Python3.6-build.log doerwalter, 2016-12-28 20:02
Python3.6-build2.log doerwalter, 2016-12-29 12:20
Messages (12)
msg284193 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2016-12-28 20:02
I'm trying to compile Python 3.6 from source on MacOS X Sierra. However it seems that the _ssl module doesn't get built. Attached is the complete output.

Note that I have openssl installed via homebrew:

~/ ▸ brew list openssl
/usr/local/Cellar/openssl/1.0.2j/bin/c_rehash
/usr/local/Cellar/openssl/1.0.2j/bin/openssl
/usr/local/Cellar/openssl/1.0.2j/include/openssl/ (75 files)
/usr/local/Cellar/openssl/1.0.2j/lib/libcrypto.1.0.0.dylib
/usr/local/Cellar/openssl/1.0.2j/lib/libssl.1.0.0.dylib
/usr/local/Cellar/openssl/1.0.2j/lib/engines/ (12 files)
/usr/local/Cellar/openssl/1.0.2j/lib/pkgconfig/ (3 files)
/usr/local/Cellar/openssl/1.0.2j/lib/ (4 other files)
/usr/local/Cellar/openssl/1.0.2j/share/man/ (1592 files)

but if I understood Mac/BuildScript/resources/ReadMe.rtf correctly, this should be irrelevant.

Anyway the resulting pip seems to be unusable:

~/ ▸ python -mpip install cx_Oracle
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting cx_Oracle
  Could not fetch URL https://pypi.python.org/simple/cx-oracle/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
msg284223 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-12-29 03:44
Did you set CFLAGS and LDFLAGS?
msg284247 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2016-12-29 11:33
No, neither CFLAGS nor LDFLAGS are set, the only "FLAGS" environment variable I have set is ARCHFLAGS='-arch x86_64' (I can't remember why). However unsetting this variable doesn't change the result.
msg284248 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-12-29 11:42
Ah, things are quite the opposite :)

You have to set CFLAGS and LDFLAGS so that setup.py can find openssl headers and libraries from Homebrew. For example:

export CFLAGS="-I/usr/local/opt/openssl/include"
export LDFLAGS="-L/usr/local/opt/openssl/lib"

The reason for that is that Apple once shipped their forked OpenSSL in older Mac OS X versions, so Homebrew can't install OpenSSL headers and libraries to public paths (/usr/local/include & /usr/local/lib). On the other hand, CPython should not use paths in /usr/local/opt/ as it's internal details of Homebrew.
msg284251 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2016-12-29 12:20
OK, I've set CFLAGS and LDFLAGS as you suggested. However the ssl module still doesn't get built. Attached is the new build log (Python3.6-build2.log)
msg284263 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-12-29 13:47
Change this line:

export CFLAGS="-I/usr/local/Cellar/openssl/1.0.2j/include/openssl"

into:

export CFLAGS="-I/usr/local/Cellar/openssl/1.0.2j/include"
msg284274 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-12-29 15:51
See the Developers Guide for more information on build dependencies and recommended ways to satisfy OpenSSL dependencies:

http://cpython-devguide.readthedocs.io/en/latest/setup.html#build-dependencies
msg284284 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2016-12-29 17:48
OK, with the fixed CFLAGS definition I do indeed get a working ssl module.

I wonder whether the link Ned posted should be put into the README file.

Anyway I think the issue can be closed. Thanks for the help!
msg290350 - (view) Author: Tommy Carstensen (Tommy.Carstensen) Date: 2017-03-24 23:28
I have the same problem as described here. How can I install Python3.6 and pip without sudo and without homebrew? I get the same error message after installation:

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
msg290453 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2017-03-25 00:40
@Tommy.Carstensen: Unfortunately, Apple has deprecated use of the system-supplied OpenSSL libraries and with the latest releases no longer supply the header files needed to build with them.  They are very old anyway.  So, you need to supply a version of them.  The easiest way is to get them from a third-party package manager like Homebrew or MacPorts but you certainly can download an OpenSSL source release from https://www.openssl.org and build the libraries themselves.  If you do not have administrator access, you will probably need to modify the OpenSSL build using at least --prefix to install to a non-system location and then rerun Python's ./configure with CFLAGS and LDFLAGS pointing to the installed location of your OpenSSL.
msg290458 - (view) Author: Tommy Carstensen (Tommy.Carstensen) Date: 2017-03-25 01:22
Thanks @ned.deily ! That did the trick for me! When installing openssl I just had to do ./config --prefix=/my/home/dir prior to doing make and make install. Then I just did two commands prior to installing python3.6:
export CFLAGS="-I/my/home/dir/include"
export LDFLAGS="-L/my/home/dir/lib"
msg303787 - (view) Author: Greg Couch (gregcouch) Date: 2017-10-05 21:11
The problem with compiling OpenSSL from source is that it doesn't know how to access the root certificates needed for verification on Mac OS X.  See issue17128 for more details.
History
Date User Action Args
2017-10-05 21:11:32gregcouchsetnosy: + gregcouch
messages: + msg303787
2017-03-25 01:22:15Tommy.Carstensensetmessages: + msg290458
2017-03-25 00:40:05ned.deilysetmessages: + msg290453
2017-03-24 23:28:03Tommy.Carstensensetnosy: + Tommy.Carstensen
messages: + msg290350
2016-12-29 17:48:41doerwaltersetmessages: + msg284284
2016-12-29 15:51:15ned.deilysetstatus: open -> closed

nosy: + ned.deily
messages: + msg284274

resolution: not a bug
stage: resolved
2016-12-29 13:47:09yan12125setmessages: + msg284263
2016-12-29 12:20:47doerwaltersetfiles: + Python3.6-build2.log

messages: + msg284251
2016-12-29 11:42:06yan12125setmessages: + msg284248
2016-12-29 11:33:15doerwaltersetmessages: + msg284247
2016-12-29 03:44:59yan12125setnosy: + yan12125
messages: + msg284223
2016-12-28 20:02:42doerwaltercreate