classification
Title: hashlib missing algorithms
Type: behavior Stage: patch review
Components: Extension Modules Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, floppymaster, gregory.p.smith
Priority: normal Keywords: patch

Created on 2019-10-21 16:00 by floppymaster, last changed 2019-10-21 20:08 by floppymaster.

Pull Requests
URL Status Linked Edit
PR 16873 closed floppymaster, 2019-10-21 16:03
Messages (5)
msg355080 - (view) Author: Mike Gilbert (floppymaster) * Date: 2019-10-21 16:00
After upgrading to Python 3.7.5, several algorithms are unavailable in the hashlib module.

This seems to have been caused by the "fix" for https://bugs.python.org/issue33936. I will submit a PR with a more appropriate change shortly.

Python 3.7.4:
>>> hashlib.algorithms_available
{'ripemd160', 'blake2s256', 'sha512', 'sha3-224', 'sha3_224', 'blake2s', 'shake_128', 'sha3-384', 'sha3_384', 'whirlpool', 'md4', 'md5', 'sha512-224', 'sha3-512', 'sha1', 'blake2b', 'sha384', 'md5-sha1', 'sha3-256', 'shake_256', 'mdc2', 'sha224', 'blake2b512', 'shake128', 'sm3', 'sha256', 'shake256', 'sha3_512', 'sha512-256', 'sha3_256'}

Python 3.7.5:
>>> hashlib.algorithms_available
{'sha224', 'sha256', 'sha3_224', 'blake2s', 'md5', 'sha3_256', 'sha384', 'sha3_512', 'sha3_384', 'shake_256', 'blake2b', 'sha512', 'shake_128', 'sha1'}
msg355089 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2019-10-21 17:46
According to OpenSSL documentation it is not necessary to call any init function. All ciphers and all digests are default options.

As of version 1.1.0 OpenSSL will automatically allocate all resources that it needs so no explicit initialisation is required. Similarly it will also automatically deinitialise as required.

OPENSSL_INIT_ADD_ALL_CIPHERS
With this option the library will automatically load and make available all libcrypto ciphers. This option is a default option.

OPENSSL_INIT_ADD_ALL_DIGESTS
With this option the library will automatically load and make available all libcrypto digests. This option is a default option.

https://www.openssl.org/docs/man1.1.0/man3/OPENSSL_init_crypto.html
msg355090 - (view) Author: Mike Gilbert (floppymaster) * Date: 2019-10-21 17:50
Then the OpenSSL documentation is wrong, or Python is subsequently doing something to disable some algorithms.
msg355091 - (view) Author: Mike Gilbert (floppymaster) * Date: 2019-10-21 18:10
I see that generate_hash_name_list() calls EVP_MD_do_all() which calls  OPENSSL_init_crypto(OPENSSL_INIT_ADD_ALL_DIGESTS, NULL);

I'm not sure why that doesn't do the job here.
msg355103 - (view) Author: Mike Gilbert (floppymaster) * Date: 2019-10-21 20:08
Ok, so this appears to be working correctly on master. Just the 3.7 branch is broken.

I think this is because we use OBJ_NAME_do_all instead of EVP_MD_do_all in 3.7.

I think backporting https://github.com/python/cpython/pull/16083 to 3.7 would resolve this.
History
Date User Action Args
2019-10-21 20:08:39floppymastersetmessages: + msg355103
2019-10-21 18:10:09floppymastersetmessages: + msg355091
2019-10-21 17:50:03floppymastersetmessages: + msg355090
2019-10-21 17:46:30christian.heimessetnosy: + gregory.p.smith, christian.heimes
messages: + msg355089
2019-10-21 16:03:35floppymastersetkeywords: + patch
stage: patch review
pull_requests: + pull_request16418
2019-10-21 16:00:04floppymastercreate