classification
Title: test_ssl: failures on OpenBSD with LibreSSL
Type: Stage:
Components: Tests Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, python-dev, rpointel, spil, vstinner
Priority: normal Keywords:

Created on 2015-01-06 10:54 by vstinner, last changed 2016-09-08 15:32 by christian.heimes. This issue is now closed.

Messages (12)
msg233514 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-06 10:54
(See also the issue #21356.)

http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x/builds/1242/steps/test/logs/stdio

======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/python-builds/3.x.borja-openbsd-x86/build/Lib/test/test_ssl.py", line 764, in test_options
    ctx.options)
AssertionError: -2130705409 != -2097150977

======================================================================
FAIL: test_openssl_version (test.test_ssl.BasicSocketTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/python-builds/3.x.borja-openbsd-x86/build/Lib/test/test_ssl.py", line 315, in test_openssl_version
    (s, t))
AssertionError: False is not true : ('LibreSSL 2.1', (2, 0, 0, 0, 0))
msg233519 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-06 11:18
changeset:   94041:87976d72fd5c
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Tue Jan 06 11:51:06 2015 +0100
files:       Lib/test/test_ssl.py
description:
test_ssl: add more debug to investigate test_openssl_version() failure on
OpenBSD with LibreSSL.
msg233534 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-06 12:43
It looks like OPENSSL_VERSION_NUMBER is consistent with OPENSSL_VERSION_INFO (version 2.1). But the OPENSSL_VERSION contains a different version (2.0). It looks like an issue in LibreSSL.

======================================================================
FAIL: test_openssl_version (test.test_ssl.BasicSocketTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/python-builds/3.x.borja-openbsd-x86/build/Lib/test/test_ssl.py", line 315, in test_openssl_version
    (s, t, hex(n)))
AssertionError: False is not true : ('LibreSSL 2.1', (2, 0, 0, 0, 0), '0x20000000')
msg233536 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-01-06 13:01
New changeset 05d7550bd2d9 by Victor Stinner in branch 'default':
Issue #23177: Document that ssl.RAND_egd() is not available with LibreSSL
https://hg.python.org/cpython/rev/05d7550bd2d9
msg233576 - (view) Author: Bernard Spil (spil) * Date: 2015-01-07 11:26
Note that the FreeBSD port modifies the OPENSSL_VERSION_NUMBER and sets the version number to 1.0.1g.
https://svnweb.freebsd.org/ports?view=revision&revision=361642
msg233578 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-07 13:54
> Note that the FreeBSD port modifies the OPENSSL_VERSION_NUMBER and sets the version number to 1.0.1g.

Maybe we should remove the test on OPENSSL_VERSION (string) for LibreSSL?
msg233632 - (view) Author: Bernard Spil (spil) * Date: 2015-01-08 09:31
LibreSSL defines in opensslv.h
#define LIBRESSL_VERSION_NUMBER 0x20000000L
#define OPENSSL_VERSION_NUMBER 0x20000000L

And FreeBSD replaces
#define OPENSSL_VERSION_NUMBER 0x1000107fL

Proper way would be to check for LIBRESSL_VERSION_NUMBER string, FreeBSD modifies the OpenSSL version number to indicate its compatibility level (as stated in commit log)
msg233633 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-08 09:32
> Proper way would be to check for LIBRESSL_VERSION_NUMBER string, FreeBSD modifies the OpenSSL version number to indicate its compatibility level (as stated in commit log)

Please see the unit test: it checks that the version number and version string are consistent. It's not the case on OpenBSD with LibreSSL (2.0 vs 2.1) nor on FreeBSD (1.0 vs 2.1).
msg234815 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-27 10:15
_ssl cannot be compiled with LibreSSL anymore (on OpenBSD 5.5) because of ALPN: see issue #23329.
msg251167 - (view) Author: Remi Pointel (rpointel) * Date: 2015-09-20 13:38
Hi,

maybe we should add the LIBRESSL_ information available in opensslv.h (as it has be done for OpenSSL):

$ grep LIBRESSL /usr/include/ssl/opensslv.h 
#define LIBRESSL_VERSION_NUMBER 0x20030000L
#define LIBRESSL_VERSION_TEXT   "LibreSSL 2.3.0"
#define OPENSSL_VERSION_TEXT    LIBRESSL_VERSION_TEXT

What do you think about that?

Remi.
msg251208 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-21 06:59
I don't care so much of issues introduced by LibreSSL, I don't understand why they broke the API. For me, it doesn't seem right to have a version different if it's a number or if it's a string: OPENSSL_VERSION_NUMBER should be consistent with OPENSSL_VERSION_INFO.

If you propose a patch for Python and it fixes test_ssl, I will apply it :-)
msg275050 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-09-08 15:32
I have fixed all libressl related test failures while I added support for OpenSSL 1.1.0.
History
Date User Action Args
2016-09-08 15:32:28christian.heimessetstatus: open -> closed

nosy: + christian.heimes
messages: + msg275050

resolution: fixed
2015-09-21 06:59:15vstinnersetmessages: + msg251208
2015-09-20 13:38:07rpointelsetmessages: + msg251167
2015-01-27 10:15:51vstinnersetmessages: + msg234815
2015-01-08 09:32:40vstinnersetmessages: + msg233633
2015-01-08 09:31:06spilsetmessages: + msg233632
2015-01-07 13:54:51vstinnersetmessages: + msg233578
2015-01-07 11:26:33spilsetnosy: + spil
messages: + msg233576
2015-01-06 13:01:08python-devsetnosy: + python-dev
messages: + msg233536
2015-01-06 12:43:41vstinnersetmessages: + msg233534
2015-01-06 11:18:11vstinnersetmessages: + msg233519
2015-01-06 10:54:16vstinnercreate