classification
Title: hashlib.pbkdf2_hmac Hash Constructor
Type: Stage:
Components: Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: alex, aronacher, christian.heimes, dstufft, egbutter, gregory.p.smith, yselivanov
Priority: normal Keywords:

Created on 2014-04-17 18:44 by aronacher, last changed 2018-01-03 17:48 by christian.heimes.

Messages (14)
msg216728 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-04-17 18:44
Is there a specific reason why hashlib.pbkdf2_hmac now has a completely inconsistent API with the rest of the stdlib?  So far the concept in both hashlib and hmac has been to accept hash constructors as parameters.

As such you would expect the API to look like this:

hashlib.pbkdf2_hmac(hashlib.sha256, b'password', b'salt', 100000)

Instead the API now is string based.

This is annoying because a lot of code already in the past exposed a pbkdf2 implementation that allows passing in a hashlib constructor by passing it to HMAC.

If such code now wants to use the stdlib pbkdf2_hmac implementation it has to special case known implementations.  If a non known implementation is found it needs to dispatch to a separate implementation of PBKDF2.

In addition to that there is no nice way to detect if a algorithm is not supported as the exception raised for an invalid algorithm is the same as an invalid parameter for the iteration count (ValueError).

I would propose to change the API to allow hash constructors to be passed in in addition to strings.
msg216730 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-04-17 18:54
This commit shows why the API is problematic: https://github.com/mitsuhiko/werkzeug/commit/c527dcbfb0ee621e9faa0a3a2873118438965800
msg216733 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-04-17 19:53
I agree that this change makes a lot of sense.
msg216741 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-04-17 21:02
A callable wouldn't work for the OpenSSL back end of PBKDF2. The function takes a digest pointer. I have to think about a solution...

Sorry for the brevity, I still don't have proper internet at home.

On 17. April 2014 22:20:36 MESZ, Yury Selivanov <report@bugs.python.org> wrote:
>
>Changes by Yury Selivanov <yselivanov.ml@gmail.com>:
>
>
>----------
>nosy: +christian.heimes, gregory.p.smith
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue21288>
>_______________________________________
msg216746 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-04-17 21:21
On 2014-04-17, 5:02 PM, Christian Heimes wrote:
> Christian Heimes added the comment:
>
> A callable wouldn't work for the OpenSSL back end of PBKDF2. The function takes a digest pointer. I have to think about a solution...
>
> Sorry for the brevity, I still don't have proper internet at home.
>

We can accept only hashlib functions, and continue passing their names 
to the OpenSSL backend. A bit ugly and limited solution (no user-defined 
hash functions) for a better looking API.

Yury
msg216747 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-04-17 21:33
> We can accept only hashlib functions, and continue passing their names 
> to the OpenSSL backend. A bit ugly and limited solution (no user-defined 
> hash functions) for a better looking API.
What I'm doing at the code for my employer is something similar.  There is a PBKDF2 implementation on top of the hmac module.  If a hashlib constructor is detected that OpenSSL implements it dispatches that to the PBKDF2 path in OpenSSL via ctypes.

Ultimately it's the same situation but it does not expose the implementation detail.
msg216748 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-04-17 21:34
I should add that we still support non OpenSSL hashers, but we go a different path.
msg216750 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2014-04-17 21:38
Armin, FWIW, I don't think it's possible to push this API change in 3.4.
msg216753 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2014-04-17 21:42
I understand that, but given that this API might be backported to 2.7 I think it should get further review.  Also, this would only be a change to the error case.  Non string arguments are currently being responded to with a TypeError.

I am not proposing to remove the string API, just also allow a hash constructor.
msg309101 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-12-27 19:56
Is anybody interested still to solve the problem? I don't see a viable way to solve the issue in a correct way (*). The hashlib API is not designed to provide sufficient information from a digestmod object. I don't want to include hacks like mapping _hashlib.openssl_sha256 function to SHA256 EVP_MD.

(*) correct means: don't call the function, always reuse the hasher's EVP_MD* context and never fall back to slow path silently.
msg309104 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2017-12-27 20:52
Yes, I'm definitely still interested in this. I still carry this hack around.
msg309107 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-12-27 22:08
Do you have a proper solution that works within the limits of PEP 247? I could not, hence the existing API.
msg309111 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2017-12-28 00:01
I have no good solution.  What I do so far is pretty much exactly what was originally reported here: https://github.com/pallets/werkzeug/blob/6922d883ba61c6884fa6cab5bfd280c5a60399af/werkzeug/security.py#L96-L104
msg309421 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-03 17:48
I'm working on a hashing algorith v2.0 PEP. New hashing constructors and modules will have an allow_optimization flag and oid attribute. Once finalized and implemented, hashlib.pbkdf2_hmac() will support digestmod and digest constructors that have allow_optimization=True and a valid OID class / module attributes.
History
Date User Action Args
2018-01-03 17:48:16christian.heimessetmessages: + msg309421
2017-12-28 00:01:18aronachersetmessages: + msg309111
2017-12-27 22:08:54christian.heimessetmessages: + msg309107
2017-12-27 20:52:54aronachersetstatus: pending -> open

messages: + msg309104
2017-12-27 19:56:08christian.heimessetstatus: open -> pending

messages: + msg309101
versions: + Python 3.7, - Python 3.4
2014-11-09 18:05:18egbuttersetnosy: + egbutter
2014-04-17 21:42:55aronachersetmessages: + msg216753
2014-04-17 21:38:33yselivanovsetmessages: + msg216750
2014-04-17 21:34:19aronachersetmessages: + msg216748
2014-04-17 21:33:30aronachersetmessages: + msg216747
versions: + Python 3.4, - Python 3.5
2014-04-17 21:21:12yselivanovsetmessages: + msg216746
2014-04-17 21:02:24christian.heimessetmessages: + msg216741
2014-04-17 20:20:35yselivanovsetnosy: + gregory.p.smith, christian.heimes
2014-04-17 20:10:36yselivanovsetnosy: + yselivanov

versions: + Python 3.5, - Python 3.4
2014-04-17 19:53:21dstufftsetnosy: + dstufft
messages: + msg216733
2014-04-17 18:54:27aronachersetmessages: + msg216730
2014-04-17 18:46:12alexsetnosy: + alex
2014-04-17 18:44:52aronachercreate