Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hashlib: OpenSSL hash detection should obey security policy #84872

Closed
tiran opened this issue May 20, 2020 · 5 comments
Closed

hashlib: OpenSSL hash detection should obey security policy #84872

tiran opened this issue May 20, 2020 · 5 comments
Assignees
@tiran
Labels
3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tiran
Copy link
Member

tiran commented May 20, 2020

BPO 40695
Nosy @gpshead, @tiran, @ned-deily, @miss-islington
PRs
  • bpo-40695: Limit hashlib builtin hash fallback #20259
  • [3.9] bpo-40695: Limit hashlib builtin hash fallback (GH-20259) #20377
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
closed_at = <Date 2021-05-22.09:17:30.971>
created_at = <Date 2020-05-20.08:07:57.604>
labels = ['type-bug', 'library', '3.9', '3.10']
title = 'hashlib: OpenSSL hash detection should obey security policy'
updated_at = <Date 2021-05-22.09:17:30.971>
user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2021-05-22.09:17:30.971>
actor = 'christian.heimes'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2021-05-22.09:17:30.971>
closer = 'christian.heimes'
components = ['Library (Lib)']
creation = <Date 2020-05-20.08:07:57.604>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 40695
keywords = ['patch']
message_count = 5.0
messages = ['369428', '369861', '369863', '394167', '394180']
nosy_count = 4.0
nosy_names = ['gregory.p.smith', 'christian.heimes', 'ned.deily', 'miss-islington']
pr_nums = ['20259', '20377']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40695'
versions = ['Python 3.9', 'Python 3.10']
    @tiran
    Copy link
    Member Author

    tiran commented May 20, 2020

    The hashlib module prefers hash implementations from OpenSSL. In case OpenSSL is not available or OpenSSL does not provide a hash algorithm, hashlib falls back to builtin implementations for MD5, SHA1, SHA2 family, SHA3/SHAKE family, and Blake2. The __get_openssl_constructor [1] function checks OpenSSL by retrieving the constructor and calling it. The calls fails if OpenSSL doesn't implement the EVP digest.

    It also fails when the EVP digest is available but blocked by a security policy. In this case it falls back to the builtin implementation. If the builtin implementation has been removed by the package builder or --with-builtin-hashlib-hashes, then Python considers the hash algorithm as broken.

    I propose to change the detection code so that Python uses OpenSSL implementation although it's blocked by the current system policy.

    Current behavior:

    $ rpm -qa openssl
openssl-1.1.1g-1.fc32.x86_64
$ /configure -C --with-builtin-hashlib-hashes=blake2
$ make -j4
$ ./python
>>> import hashlib
ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/root/cpython/Lib/hashlib.py", line 131, in __get_openssl_constructor
    f()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/cpython/Lib/hashlib.py", line 251, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/root/cpython/Lib/hashlib.py", line 135, in __get_openssl_constructor
    return __get_builtin_constructor(name)
  File "/root/cpython/Lib/hashlib.py", line 118, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'hashlib' has no attribute 'md5'

    Proposed behavior:

    $ ./python
>>> import hashlib
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
>>> hashlib.md5(usedforsecurity=False)
<md5 _hashlib.HASH object @ 0x7fb9d44b9b30>

    Related issue:

    bpo-9216 added the new hash constructor argument "usedforsecurity".
    bpo-40637 added a new configure option --with-builtin-hashlib-hashes

    [1]

    cpython/Lib/hashlib.py

    Lines 121 to 135 in 97fe9cf

    def __get_openssl_constructor(name):
    if name in __block_openssl_constructor:
    # Prefer our blake2 and sha3 implementation.
    return __get_builtin_constructor(name)
    try:
    # MD5, SHA1, and SHA2 are in all supported OpenSSL versions
    # SHA3/shake are available in OpenSSL 1.1.1+
    f = getattr(_hashlib, 'openssl_' + name)
    # Allow the C module to raise ValueError. The function will be
    # defined but the hash not actually available thanks to OpenSSL.
    f()
    # Use the C function directly (very fast)
    return f
    except (AttributeError, ValueError):
    return __get_builtin_constructor(name)

    @tiran tiran added 3.9 only security fixes 3.10 only security fixes labels May 20, 2020
    @tiran tiran self-assigned this May 20, 2020
    @tiran tiran added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error 3.9 only security fixes 3.10 only security fixes labels May 20, 2020
    @tiran tiran self-assigned this May 20, 2020
    @tiran tiran added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels May 20, 2020
    @miss-islington
    Copy link
    Contributor

    New changeset 4cc2f93 by Christian Heimes in branch 'master':
    bpo-40695: Limit hashlib builtin hash fallback (GH-20259)
    4cc2f93

    @miss-islington
    Copy link
    Contributor

    New changeset 7015823 by Miss Islington (bot) in branch '3.9':
    bpo-40695: Limit hashlib builtin hash fallback (GH-20259)
    7015823

    @ned-deily
    Copy link
    Member

    Is there anything more that needs to be done for this issue?

    @tiran
    Copy link
    Member Author

    tiran commented May 22, 2021

    No, nothing left to do. Thanks for the ping!

    @tiran tiran closed this as completed May 22, 2021
    @tiran tiran closed this as completed May 22, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @tiran tiran

    Labels
    3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @tiran @ned-deily @miss-islington