This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: hashlib.algorithms_available lists algorithms that are not available in OpenSSL 3.0 default provider
Type: behavior Stage: patch review
Components: Extension Modules, Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, miss-islington
Priority: normal Keywords: patch

Created on 2022-03-23 14:28 by christian.heimes, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 32076 merged christian.heimes, 2022-03-23 14:33
PR 32084 merged miss-islington, 2022-03-23 20:30
PR 32085 merged christian.heimes, 2022-03-23 20:31
Messages (6)
msg415877 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-03-23 14:28
Hubert Kario wrote in https://bugzilla.redhat.com/show_bug.cgi?id=2054702

Description of problem:
The hashlib.algorithms_available set includes algorithms like ripemd160 and whirlpool, those algorithms are not usable unless openssl legacy provider is loaded. Since it's not loaded, and the hashlib module won't load it, any attempt to use them fails.

Version-Release number of selected component (if applicable):
python3-3.9.10-1.el9.x86_64
openssl-3.0.1-5.el9.x86_64

How reproducible:
always

Steps to Reproduce:
0. start python3
1. from hashlib import algorithms_available 
2. algorithms_available
3. import hashlib
4. a = {(name, hashlib.new(name).digest_size) for name in algorithms_available}

Actual results:
{'sha3_384', 'blake2s', 'sha384', 'sha512_224', 'md5', 'sha3_512', 'md5-sha1', 'sha3_256', 'shake_128', 'sm3', 'sha256', 'sha512', 'sha1', 'shake_256', 'blake2b', 'whirlpool', 'sha512_256', 'sha3_224', 'sha224', 'ripemd160', 'md4'}

Traceback (most recent call last):
  File "/usr/lib64/python3.9/hashlib.py", line 164, in __hash_new
    return _hashlib.new(name, data, **kwargs)
ValueError: [digital envelope routines] unsupported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <setcomp>
  File "/usr/lib64/python3.9/hashlib.py", line 170, in __hash_new
    return __get_builtin_constructor(name)(data)
  File "/usr/lib64/python3.9/hashlib.py", line 127, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type whirlpool


Expected results:
{'sha3_384', 'blake2s', 'sha384', 'sha512_224', 'md5', 'sha3_512', 'md5-sha1', 'sha3_256', 'shake_128', 'sm3', 'sha256', 'sha512', 'sha1', 'shake_256', 'blake2b', 'sha512_256', 'sha3_224', 'sha224'}

{('blake2b', 64), ('sha512', 64), ('md5-sha1', 36), ('sha3_512', 64), ('md5', 16), ('sha224', 28), ('shake_128', 0), ('sm3', 32), ('blake2s', 32), ('sha1', 20), ('shake_256', 0), ('sha512_256', 32), ('sha3_224', 28), ('sha3_256', 32), ('sha3_384', 48), ('sha384', 48), ('sha256', 32), ('sha512_224', 28)}

Additional info:
If the legacy provider is loaded, then the algorithms should be listed and should work.

It may be caused by Python using the deprecated EVP_MD_do_all() method instead of the EVP_MD_do_all_provided() method
msg415878 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-03-23 14:38
Hubert's suggested solution EVP_MD_do_all_provided() worked almost straight forward. The function signature is a bit different and I got "undefined" in the result set. Filtering out NID_undef got right of it.
msg415880 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-03-23 15:50
$ ./python Tools/ssl/multissltests.py --openssl 3.0.2 --steps modules
$ ./python -c "import hashlib; print(hashlib.algorithms_available)"
{'blake2b', 'sha512', 'sm3', 'shake_128', 'md5', 'sha3_256', 'sha224', 'sha512_224', 'sha3_384', 'sha384', 'md5-sha1', 'sha3_224', 'shake_256', 'sha3_512', 'sha512_256', 'sha1', 'sha256', 'blake2s'}
msg415907 - (view) Author: miss-islington (miss-islington) Date: 2022-03-23 20:30
New changeset 48e2010d92076b472922fa632fffc98ee150004f by Christian Heimes in branch 'main':
bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076)
https://github.com/python/cpython/commit/48e2010d92076b472922fa632fffc98ee150004f
msg415909 - (view) Author: miss-islington (miss-islington) Date: 2022-03-23 20:58
New changeset ec3589f59d2c8456591f33656639bcc303eb7bd5 by Miss Islington (bot) in branch '3.9':
bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076)
https://github.com/python/cpython/commit/ec3589f59d2c8456591f33656639bcc303eb7bd5
msg415912 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-03-23 21:15
New changeset 1b6acaad9a18b2498386c60f24351ab749061e3a by Christian Heimes in branch '3.10':
[3.10] bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076) (GH-32085)
https://github.com/python/cpython/commit/1b6acaad9a18b2498386c60f24351ab749061e3a
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91257
2022-03-23 21:15:34christian.heimessetmessages: + msg415912
2022-03-23 20:58:06miss-islingtonsetmessages: + msg415909
2022-03-23 20:31:17christian.heimessetpull_requests: + pull_request30172
2022-03-23 20:30:22miss-islingtonsetpull_requests: + pull_request30171
2022-03-23 20:30:17miss-islingtonsetnosy: + miss-islington
messages: + msg415907
2022-03-23 15:50:38christian.heimessetmessages: + msg415880
2022-03-23 14:38:17christian.heimessetmessages: + msg415878
2022-03-23 14:33:20christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request30164
2022-03-23 14:28:49christian.heimescreate