This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Cannot tune scrypt with large enough parameters
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Gle, christian.heimes, gregory.p.smith
Priority: normal Keywords:

Created on 2020-03-16 16:17 by Gle, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
compare.py Gle, 2020-03-16 16:17 Run scrypt KDF from hashlib and from cryptography module for comparison.
Messages (4)
msg364334 - (view) Author: (Gle) Date: 2020-03-16 16:17
I can use scrypt KDF with the cryptography module
https://cryptography.io/en/latest/hazmat/primitives/key-derivation-functions/#cryptography.hazmat.primitives.kdf.scrypt.Scrypt
with large parameters (n=2**20, r=16, p=1)

On the other hand, using scrypt KDF from hashlib with the same parameters yields "Invalid combination of n, r, p, maxmem" (I use maxmem=0).

Shouldn't they behave the same ? As they both seem to be wrappers around OpenSSL ?

I've also included a set of functioning parameters as hashlib's scrypt works fine on small parameter values.

Notice that the output from hashlib's scrypt is different than the output from the cryptography module. Shouldn't they be the same ? (I'm no cryptography expert)

I would really like to be able to use scrypt for hardened password hashing using only python standard library's hashlib. Maybe I'm missing something ?

Python is great ! Thanks for all the good work !
msg364479 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-03-17 21:34
Your parameter selection requires about 64 MB of memory (n * 2 * r * 64).
As documented maxmem=0 defaults to 32 MB of maximum memory in OpenSSL 1.1.x. OpenSSL needs a bit of internal memory for book keeping and other stuff, so you need maxmem=65*1024*1024 for your parameter set.

PyCA cryptography has a default maxmem of sys.maxsize // 2, that's a couple of Exabyte of RAM on a 64bit system.
msg364513 - (view) Author: (Gle) Date: 2020-03-18 11:32
Alright, I understand the difference in behaviour now. Thanks a lot for the clear explanation !

Would be nice to have something like:
"""maxmem must be greater than (n * 2 * r * 64) plus a bit of internal
   memory for OpenSSL book keeping.
   Basically, set maxmem = (n * 2 * r * 65)
"""
in the documentation.

Thanks again, and sorry to have bothered you with this non-bug.
Have a happy day !
msg364516 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-03-18 12:26
PS: You are getting a different output because you are feeding a different input to hashlib.scrypt(). The first parameter is the password, not password + salt.
History
Date User Action Args
2022-04-11 14:59:28adminsetgithub: 84160
2020-03-18 12:26:26christian.heimessetmessages: + msg364516
2020-03-18 11:32:56Glesetstatus: open -> closed
resolution: not a bug
messages: + msg364513

stage: resolved
2020-03-17 21:34:36christian.heimessetmessages: + msg364479
2020-03-16 16:17:28Glecreate