classification
Title: _ssl cannot be compiled with LibreSSL anymore (on OpenBSD 5.5) because of ALPN
Type: compile error Stage: resolved
Components: Extension Modules Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, benjamin.peterson, martin.panter, python-dev, rpointel, spil, vstinner
Priority: normal Keywords: patch

Created on 2015-01-27 10:14 by vstinner, last changed 2015-09-27 07:09 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
python_have_alpn.diff rpointel, 2015-09-23 19:05 review
Messages (11)
msg234814 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-27 10:14
Recently, the issue #20188 "ALPN support for TLS" was fixed. The problem is that the check for the ALPN feature doesn't work with LibreSSL:

/* ALPN added in OpenSSL 1.0.2 */
#if OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined(OPENSSL_NO_TLSEXT)
# define HAVE_ALPN
#endif

On the buildbot OpenBSD 5.5 with LibreSSL, OPENSSL_VERSION_NUMBER is 2.x instead of 1.0.x. See also the issue #23177.

A workaround would be to disable the feature if LIBRESSL_VERSION_NUMBER is defined.

http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x/builds/1333/steps/test/logs/stdio

 using PTY: False
running build
running build_ext
ldd: /usr/lib/libreadline.a: not an ELF executable
INFO: Can't locate Tcl/Tk libs and/or headers
building '_ssl' extension
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.o
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c: In function 'PySSL_selected_alpn_protocol':
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:1481: warning: implicit declaration of function 'SSL_get0_alpn_selected'
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c: In function '_set_alpn_protocols':
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:2404: warning: implicit declaration of function 'SSL_CTX_set_alpn_protos'
/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.c:2406: warning: implicit declaration of function 'SSL_CTX_set_alpn_select_cb'
gcc -pthread -shared -fPIC build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ssl.o -L/usr/local/lib -lssl -lcrypto -o build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so
building '_ctypes' extension
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.c -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.o -Wall -fexceptions
gcc -pthread -fPIC -fno-strict-aliasing -Wsign-compare -g -O0 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi/include -Ibuild/temp.openbsd-5.6-i386-3.5-pydebug/libffi -I/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src -I./Include -I. -IInclude -I/usr/local/include -I/home/python-builds/3.x.borja-openbsd-x86/build/Include -I/home/python-builds/3.x.borja-openbsd-x86/build -c /home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.S -o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.o -Wall -fexceptions
gcc -pthread -shared -fPIC build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/_ctypes.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callbacks.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/callproc.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/stgdict.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/cfield.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/prep_cif.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/closures.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/ffi.o build/temp.openbsd-5.6-i386-3.5-pydebug/home/python-builds/3.x.borja-openbsd-x86/build/Modules/_ctypes/libffi/src/x86/freebsd.o -L/usr/local/lib -o build/lib.openbsd-5.6-i386-3.5-pydebug/_ctypes.so
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_CTX_set_alpn_protos'
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_get0_alpn_selected'
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ssl.so: undefined symbol 'SSL_CTX_set_alpn_select_cb'
*** WARNING: renaming "_ssl" since importing it failed: Cannot load specified object
./python:build/lib.openbsd-5.6-i386-3.5-pydebug/_ctypes.so: undefined symbol 'ffi_call_win32'
*** WARNING: renaming "_ctypes" since importing it failed: Cannot load specified object

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_tkinter              ossaudiodev           spwd               
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Following modules built successfully but were removed because they could not be imported:
_ctypes               _ssl
msg234825 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-01-27 16:10
New changeset 53e94a687570 by Benjamin Peterson in branch 'default':
disable ALPN on LibreSSL, which has a large version number, but not ALPN support (closes #23329)
https://hg.python.org/cpython/rev/53e94a687570

New changeset f7fd2776e80d by Benjamin Peterson in branch '2.7':
disable ALPN on LibreSSL, which has a large version number, but not ALPN support (closes #23329)
https://hg.python.org/cpython/rev/f7fd2776e80d
msg234827 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-27 16:38
Cool, the issue looks like the issue has been fixed: the _ssl module can be build again.
http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x/builds/1334/steps/compile/logs/stdio

Thanks for the quick fix.
msg234828 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-01-27 16:56
(Thanks for pointing out the problem and the fix.)
msg251321 - (view) Author: Bernard Spil (spil) * Date: 2015-09-22 13:48
ALPN was removed originally but added again later
http://marc.info/?l=openbsd-announce&m=142193407304782
msg251339 - (view) Author: Remi Pointel (rpointel) * Date: 2015-09-22 18:48
Maybe we could check if the functionality is available instead of checking a version? What do you think about that?
msg251408 - (view) Author: Bernard Spil (spil) * Date: 2015-09-23 06:59
Checking if the method is actually defined is the smart way to go

From include/openssl/tls1.h
# define TLSEXT_TYPE_application_layer_protocol_negotiation

So #ifdef TLSEXT_TYPE_application_layer_protocol_negotiation
would show you if the feature is available regardless of the implementation (Open|Boring|Libre)SSL
msg251439 - (view) Author: Remi Pointel (rpointel) * Date: 2015-09-23 19:05
With this patch it works fine on OpenBSD with LibreSSL.
Could you test on a Linux machine with OpenSSL supporting ALPN please?

Thanks spil@ for the idea.
msg251620 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-09-25 23:08
I just tested your patch on Arch Linux with the default 3.6 Python branch. The test suite passed happily and all seems to be well:

Python 3.6.0a0 (python_have_alpn.diff qbase qtip tip:eb8ee60ace13, Sep 25 2015, 22:53:15) 
[GCC 5.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.HAS_ALPN
True
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2d 9 Jul 2015'
msg251652 - (view) Author: Remi Pointel (rpointel) * Date: 2015-09-26 15:17
Good, I think we should test on a machine with a version of openssl that does not support ALPN to be sure.
msg251685 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-09-27 07:09
New changeset 38a5b0f6531b by Benjamin Peterson in branch '3.5':
detect alpn by feature flag not openssl version (closes #23329)
https://hg.python.org/cpython/rev/38a5b0f6531b

New changeset 747996431c7e by Benjamin Peterson in branch 'default':
merge 3.5 (#23329)
https://hg.python.org/cpython/rev/747996431c7e
History
Date User Action Args
2015-09-27 07:09:17python-devsetstatus: open -> closed
resolution: fixed
messages: + msg251685
2015-09-26 15:17:15rpointelsetmessages: + msg251652
2015-09-25 23:08:34martin.pantersetnosy: + martin.panter
messages: + msg251620
2015-09-23 19:57:07vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
2015-09-23 19:05:47rpointelsetfiles: + python_have_alpn.diff
keywords: + patch
messages: + msg251439
2015-09-23 06:59:31spilsetmessages: + msg251408
2015-09-22 18:48:59rpointelsetmessages: + msg251339
2015-09-22 13:48:23spilsetmessages: + msg251321
2015-01-27 16:56:09benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg234828
2015-01-27 16:38:15vstinnersetmessages: + msg234827
2015-01-27 16:10:42python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg234825

resolution: fixed
stage: resolved
2015-01-27 10:50:26Arfreversetnosy: + Arfrever
2015-01-27 10:14:52vstinnercreate