classification
Title: Can no longer specify OpenSSL locations with CPPFLAGS / LDFLAGS ?
Type: enhancement Stage: needs patch
Components: Build Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Alex.Willmer, barry, christian.heimes, ned.deily, yselivanov
Priority: high Keywords: patch

Created on 2018-01-28 15:11 by yselivanov, last changed 2018-05-24 06:04 by ned.deily.

Pull Requests
URL Status Linked Edit
PR 5388 closed yselivanov, 2018-01-28 15:17
Messages (8)
msg310953 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-01-28 15:13
Not sure if we should do it or not:

1. MacPorts maintainers will create a port file for Python 3.7 and they will just configure it with `configure --with-openssl=$PREFIX", where PREFIX is /opt/local by default.

2. This, however, would certainly improve the experience of manually building CPython on MacOS with macports (this would directly benefit guys like me).
msg310967 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-28 16:55
The good news is that, since Christian's recent enhancements in Issue32598 to have autoconf use pkg-config to detect OpenSSL header and lib files, configure will now automatically find the MacPorts OpenSSL as long as the MacPorts bin directory (typically /opt/local/bin) is in the shell PATH when you run configure.  MacPorts supplies pkg-config and .pc files for various packages it installs including OpenSSL. (Apple doesn't ship a copy of pkg-config with macOS).  I found this out by accident; thanks, Christian!  The current status quo is also more robust in that MacPorts is not necessarily installed in /opt/local, a problem with PR 5388 as it stands.  I install MacPorts from source to a non-standard location so it would be good for someone to verify that the current 3.7 configure works with a default (/opt/local) binary install of MacPorts.
msg310968 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-01-28 17:04
> The good news is that, since Christian's recent enhancements in Issue32598 to have autoconf use pkg-config to detect OpenSSL header and lib files, configure will now automatically find the MacPorts OpenSSL as long as the MacPorts bin directory (typically /opt/local/bin) is in the shell PATH when you run configure.

/opt/local/bin *is* in my shell's PATH and `./configure' still fails to auto-discover OpenSSL:

    ~ » echo $PATH
    /Users/yury/dev/bin /opt/local/bin /opt/local/sbin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /usr/local/share/dotnet /Users/yury/dev/sys-venvs/fish/bin
msg310969 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-01-28 17:19
OK, so the trick is to first install pkg-config:

   sudo port install pkgconfig
   ./configure
   make
   python -m ssl  # works

I'll close the PR then, but we need to document this trick with pkg-config somewhere.
msg310971 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-28 17:32
Good!  Yes, that should go into the devguide.  Unfortunately, it only works with 3.7+, so the CFLAGS/LDFLAGS stuff is still needed for 3.6- and 2.7.
msg311152 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-29 17:45
Hmm, I spoke a bit too soon. I now see one side effect of Christian's change is to break the suggested code in the devguide for using a local or private version of OpenSSL on macOS at least, e.g. supplying the paths via CPPFLAGS and LDFLAGS.  If there is a pkg-config available on the path that supplies the same values, the net effect is the same but having pkg-config available wasn't a requirement previously and should not now be.  We'll probably need to fix this in a more general way than PR 5388 proposed.  I'll try to take a look at it before beta 1 freeze but, if not, it should be fixed for beta 2.
msg311213 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-30 02:06
Can we also cover Brew?

Here's the dumb little configure wrapper I use:

```
#!/bin/sh

export CPPFLAGS="-I$(brew --prefix sqlite3)/include -I$(brew --prefix zlib)/include"
export LDFLAGS="-L$(brew --prefix sqlite3)/lib -L$(brew --prefix zlib)/lib"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

./configure
```
msg317532 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-05-24 06:04
At the PyCon US sprints, we looked into this and updated the Developer's Guide with suggestions for how to configure a build on macOS with either Homebrew or MacPorts for either 3.7+ or earlier.  It is not an ideal solution so I want to keep this open for now and reassign to me but it should no longer be a "deferred blocker".

For reference, the devguide now recomments this:

(Homebrew)

$ brew install openssl xz
and configure python versions >= 3.7:

./configure --with-pydebug --with-openssl=$(brew --prefix openssl)
or configure python versions < 3.7:

$ CPPFLAGS="-I$(brew --prefix openssl)/include" \
  LDFLAGS="-L$(brew --prefix openssl)/lib" \
  ./configure --with-pydebug

(MacPorts)

$ sudo port install pkgconfig openssl xz
and configure:

$ CPPFLAGS="-I/opt/local/include" \
  LDFLAGS="-L/opt/local/lib" \
  ./configure --with-pydebug
History
Date User Action Args
2018-05-24 06:04:36ned.deilysetpriority: deferred blocker -> high
versions: + Python 3.8
messages: + msg317532

assignee: christian.heimes -> ned.deily
components: + Build, - Cross-Build
stage: patch review -> needs patch
2018-01-31 01:39:49josh.rsettitle: Can no longer specify OpenSLL locations with CPPFLAGS / LDFLAGS ? -> Can no longer specify OpenSSL locations with CPPFLAGS / LDFLAGS ?
2018-01-30 02:06:59barrysetmessages: + msg311213
2018-01-29 17:45:20ned.deilysetpriority: normal -> deferred blocker

messages: + msg311152
title: macos/configure: Discover OpenSSL when installed with MacPorts -> Can no longer specify OpenSLL locations with CPPFLAGS / LDFLAGS ?
2018-01-28 17:40:22barrysetnosy: + barry
2018-01-28 17:32:41ned.deilysetmessages: + msg310971
2018-01-28 17:19:03yselivanovsetmessages: + msg310969
2018-01-28 17:04:42yselivanovsetmessages: + msg310968
2018-01-28 16:55:27ned.deilysetmessages: + msg310967
2018-01-28 15:29:45christian.heimessetassignee: christian.heimes
2018-01-28 15:17:14yselivanovsetkeywords: + patch
stage: patch review
pull_requests: + pull_request5223
2018-01-28 15:13:41yselivanovsetmessages: + msg310953
2018-01-28 15:11:23yselivanovcreate