Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support LibreSSL (instead of OpenSSL): make RAND_egd optional #65555

Closed
EddBarrett mannequin opened this issue Apr 26, 2014 · 25 comments
Closed

Support LibreSSL (instead of OpenSSL): make RAND_egd optional #65555

EddBarrett mannequin opened this issue Apr 26, 2014 · 25 comments
Labels
build The build process and cross-build

Comments

@EddBarrett
Copy link
Mannequin

EddBarrett mannequin commented Apr 26, 2014

BPO 21356
Nosy @pitrou, @vstinner, @giampaolo, @tiran, @florentx, @mgorny, @koobs, @dstufft
Files
  • patch-Lib_ssl.py: Make RAND_egd support automatic
  • patch-Modules__ssl.c: Make RAND_egd support automatic
  • patch-configure.ac: Make RAND_egd support automatic
  • test_ssl.log: Output of test_ssl.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2015-01-06.13:01:58.158>
created_at = <Date 2014-04-26.14:21:36.056>
labels = ['build']
title = 'Support LibreSSL (instead of OpenSSL): make RAND_egd optional'
updated_at = <Date 2015-01-06.13:01:58.157>
user = 'https://bugs.python.org/EddBarrett'

    bugs.python.org fields:

    activity = <Date 2015-01-06.13:01:58.157>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-01-06.13:01:58.158>
closer = 'vstinner'
components = ['Build']
creation = <Date 2014-04-26.14:21:36.056>
creator = 'Edd.Barrett'
dependencies = []
files = ['37300', '37301', '37302', '37304']
hgrepos = []
issue_num = 21356
keywords = []
message_count = 25.0
messages = ['217198', '217199', '226355', '226357', '226819', '226832', '231426', '231428', '231462', '231471', '231797', '231798', '231799', '231801', '231802', '231803', '231804', '231807', '231808', '231809', '231812', '231814', '231839', '233535', '233537']
nosy_count = 15.0
nosy_names = ['janssen', 'pitrou', 'vstinner', 'giampaolo.rodola', 'christian.heimes', 'flox', 'polymorphm', 'mgorny', 'python-dev', 'rpointel', 'oberstet', 'koobs', 'dstufft', 'Edd.Barrett', 'spil']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue21356'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']
    @EddBarrett
    Copy link
    Mannequin Author

    EddBarrett mannequin commented Apr 26, 2014

    Hi,

    I'm sure you have heard about OpenBSD's LibreSSL fork of OpenSSL. There has been a lot of code reorganisation and removal. One function which was removed RAND_egd() breaks the CPython build. CPython no longer builds on OpenBSD.

    I have submitted a patch against PyPy already. The application library part of the change can probably be re-used since PyPy borrows CPython's application-level standard library (including the ssl and socket module). However, for the interpreter level change, the build system will probably have to be hacked. We need to check for the existence of RAND_egd() at configure time and only build in support if the function is found.

    The PyPy patch (and some discussion) is here:
    https://bitbucket.org/pypy/pypy/pull-request/233/fix-translation-for-libressl-and-fix-ssl/diff#comment-1744605

    I may have a go at doing this myself (for Python-2.7 at least) if no-one steps up in the meantime; for now just making the CPython devs aware.

    Thanks

    @EddBarrett EddBarrett mannequin added the build The build process and cross-build label Apr 26, 2014
    @pitrou
    Copy link
    Member

    pitrou commented Apr 26, 2014

    This should wait until the LibreSSL API stabilizes.

    Regardless, I think we should consider deprecating RAND_egd(). The Entropy Gathering Daemon doesn't seem to have seen a release for more than 10 years... (http://sourceforge.net/projects/egd/files/)

    @vstinner vstinner changed the title LibreSSL/RAND_egd fix needed. Support LibreSSL (instead of OpenSSL): make RAND_egd optional Apr 27, 2014
    @vstinner
    Copy link
    Member

    vstinner commented Sep 4, 2014

    The PyPy patch (and some discussion) is here:

    Your patch checks at runtime if libssl comes with RAND_egd:

       HAVE_OPENSSL_RAND_EGD = rffi_platform.Has('RAND_egd')

    In CPython, the _ssl module is compiled in C. How can we check if libssl provides RAND_egd() or not at compile time?

    Is there a way to check if libssl is OpenSSL or LibreSSL?

    @vstinner
    Copy link
    Member

    vstinner commented Sep 4, 2014

    Related discussion:
    http://marc.info/?l=openbsd-tech&m=140512043210089&w=2

    The answer for Python is:
    "your package maintainers and ask them to configure these software without egd support."

    @mgorny
    Copy link
    Mannequin

    mgorny mannequin commented Sep 12, 2014

    In CPython, the _ssl module is compiled in C. How can we check if libssl provides RAND_egd() or not at compile time?

    How about... checking whether the function is provided? Unless I'm missing some major point, AC_CHECK_FUNC should be good enough.

    Is there a way to check if libssl is OpenSSL or LibreSSL?

    Why would you want to do that? Do you want to make silly assumptions on API depending on provider name, and then add extra conditionals for versions?

    @pitrou
    Copy link
    Member

    pitrou commented Sep 12, 2014

    Unless I'm missing some major point, AC_CHECK_FUNC should be good enough.

    Building extension modules such as ssl doesn't involve autoconf.

    Do you want to make silly assumptions on API depending on provider name, and then add extra conditionals for versions?

    Arguably it would be better if LibreSSL exposed the same API as OpenSSL. We're not responsible for the discrepancy here.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 20, 2014

    EGD was only necessary for some commercial UNIX systems, versions that needed it all reached end of life. It no longer makes sense to have any code referring to it.

        EGD needed until        OS release date

    IRIX 6.5.19 feb 2003
    Solaris 2.6 jul 1997
    AIX 5.2 oct 2002
    Tru64 5.1B sep 2002
    HP-UX 11i v2 sep 2003

    Please check OpenBSD's patches to remove EGD support from Python for many versions.
    http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/python/2.7/patches/patch-Lib_ssl_py
    http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/python/3.4/patches/patch-Lib_ssl_py
    http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/lang/python/3.4/patches/patch-Lib_ssl_py

    Alternatively see Gentoo's LibreSSL changes https://github.com/Sp1l/libressl/tree/master/dev-lang/python

    @vstinner
    Copy link
    Member

    We don't drop feature in minor releases, we are working hard to maintain the backward compatibility.

    We may only disable RAND_egd if Python is compiled/linked to LibreSSL. So the check should probably be dynamic.

    @pitrou
    Copy link
    Member

    pitrou commented Nov 21, 2014

    We're still willing to fix this if someone tells us how to test for LibreSSL in C code.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 21, 2014

    Hi,

    I think this can be found in LibreSSL's opensslv.h
    An ifdef LIBRESSL_VERSION_NUMBER should work

    See https://github.com/libressl-portable/openbsd/blob/master/src/lib/libssl/src/crypto/opensslv.h

    _ssl.c includes crypto.h which in turn includes opensslv.h so checking for LIBRESSL_VERSION_NUMBER should provide the correct check.

    Attached patch does this in C whereas it should be checked for in configure and disabled with a HAS_RAND_egd
    Have not figured out how to do this conditionally in Lib/ssl.py yet

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 28, 2014

    When configure is called with correct LDFLAGS and CPPFLAGS for LibreSSL these patches to configure, Modules/_ssl.c and Lib/_ssl.py will detect not having RAND_egd support in OpenSSL and make the build succeed.

    @vstinner
    Copy link
    Member

    patch-configure.ac:
    -AC_DEFINE(__BSD_VISIBLE, 1, [Define on FreeBSD to activate all library features])

    Why do you remove this define?

    @pitrou
    Copy link
    Member

    pitrou commented Nov 28, 2014

    I thikn RAND_egd() should probably raise NotImplementedError if the function isn't exposed by the ssl library.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 28, 2014

    Victor: That is a change that has been implemented in the downstream port to fix wxPython, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192365 this ended up in this patch as my primary objective was to fix it for the FreeBSD port.

    Antoine: Sorry, I'm not a python dev... I'm willing to do the work if you can provide the guidance... This was merely a "works-for-me(TM)" patch. Since nothing actually uses egd any longer I would not spend to much effort on it. The odds of anyone requiring EGD support _and_ using LibreSSL are negligable. EGD is last centuries technology, there's no sense in mixing that with current tech.

    @vstinner
    Copy link
    Member

    Victor: That is a change that has been implemented in the downstream port to fix wxPython, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192365 this ended up in this patch as my primary objective was to fix it for the FreeBSD port.

    It looks unrelated to LibreSSL, please split your patch in two parts and open a new issue for the wxPython fix.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 28, 2014

    Remove https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192365 patch from this patch-set

    @vstinner
    Copy link
    Member

    I thikn RAND_egd() should probably raise NotImplementedError if the function isn't exposed by the ssl library.

    I would prefer to follow the model of the os module: don't declare a function if it is not supported by the OS.

    @pitrou
    Copy link
    Member

    pitrou commented Nov 28, 2014

    I would prefer to follow the model of the os module: don't declare a function if it is not supported by the OS.

    I don't have any strong feelings, so let's do it like that. RAND_egd() isn't useful anyway.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Nov 28, 2014

    New changeset 6f23bc5d480e by Victor Stinner in branch 'default':
    Issue bpo-21356: Make ssl.RAND_egd() optional to support LibreSSL. The
    https://hg.python.org/cpython/rev/6f23bc5d480e

    @vstinner
    Copy link
    Member

    Ok, here is a first commit to try to support LibreSSL in Python 3.5.

    Can someone please test to compile Python 3.5 with LibreSSL and run the test suite (at least test_ssl) to check that everything is fine? If you confirm that the change is correct, I will backport it to Python 2.7 and 3.4. Please mention your version of LibreSSL, OS and OS version in your feedback. LibreSSL has different releases: 2.0 to 2.1.1. Which one was embeded in OpenBSD 5.6?
    http://www.libressl.org/

    Bernard Spil's patches don't apply on Python 3.5, I guess that they were written for Python 2.7. I also fixed test_ssl.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 28, 2014

    FAILED (failures=2, errors=2, skipped=5)
    That is OK, as these 2 tests should fail with LibreSSL since SSLv2 and SSLv3 support has been removed from LibreSSL.

    ERROR: test_protocol_sslv23 (main.ThreadedTests)
    ERROR: test_protocol_sslv3 (main.ThreadedTests)

    @vstinner
    Copy link
    Member

    That is OK, as these 2 tests should fail with LibreSSL since SSLv2 and SSLv3 support has been removed from LibreSSL.

    See the issue bpo-22935.

    I prefer to wait until this issue is fixed in Python 3.5, and that test_ssl pass on your PC, before backporting this change into Python 2.7 & 3.4.

    @spil
    Copy link
    Mannequin

    spil mannequin commented Nov 28, 2014

    Merged the patch from haypo back into the FreeBSD port for 2.7 at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192511
    In the process I discovered during test_ssl that I had to patch Lib/socket.py as well to make RAND_egd conditional

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jan 6, 2015

    New changeset eddcb6671a48 by Victor Stinner in branch '2.7':
    Issue bpo-21356: Make ssl.RAND_egd() optional to support LibreSSL. The
    https://hg.python.org/cpython/rev/eddcb6671a48

    New changeset 7f82f50fdad0 by Victor Stinner in branch '3.4':
    Issue bpo-21356: Make ssl.RAND_egd() optional to support LibreSSL. The
    https://hg.python.org/cpython/rev/7f82f50fdad0

    @vstinner
    Copy link
    Member

    vstinner commented Jan 6, 2015

    Ok, Python 2.7, 3.4 and 3.5 can now be *compiled* with LibreSSL.

    There are still issues with LibreSSL: see the new issue bpo-23177.

    @vstinner vstinner closed this as completed Jan 6, 2015
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    build The build process and cross-build
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @pitrou