This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: secrets should use getrandom() on Linux
Type: security Stage:
Components: Versions: Python 3.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: JelleZijlstra, Lukasa, brett.cannon, ncoghlan, ned.deily, steven.daprano, tim.peters
Priority: deferred blocker Keywords:

Created on 2016-06-10 19:56 by dstufft, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (10)
msg268147 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-10 19:56
In 3.5.0 and 3.5.1 os.urandom will, where available, use the getrandom() to block rather than get insecure random from the urandom pool on Linux. In 3.5.2 this change is reverted so that os.urandom will return possibly predictable random numbers instead of blocking waiting for /dev/urandom to be intialized.

However, secrets.py is a module which is explicitly for getting cryptographically safe data for, and reverting that change means that the functions in this module are no longer cryptographically safe on Linux if they are called early enough in the boot process. Thus, secrets.py should be modified so that it no longer uses os.urandom on systems where there is a better source of randomness available-- namely getrandom() set to block on Linux.
msg268149 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-10 19:58
As an additional aside, the documentation of secrets references the documentation of random.py for it's secrets.SystemRandom class, however random.py docouments random.SystemRandom as using os.urandom.

So the documentation for secrets.SystemRandom should be disentangled from the documentation of random.SystemRandom so that it can choose to use the stronger randomness function (or random.SystemRandom should be adapted to use the stronger randomness function, or both things).
msg268152 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2016-06-10 20:15
It was a primary purpose of `secrets` to be a place where security best practices could be implemented, and changed over time, with no concern about backward compatibility for people who don't use it.

So if `secrets` needs to supply a class with all the methods of random.Random, it should derive its own subclass (or derive from random.SystemRandom, and override the only two methods that explicitly invoke _urandom() - everything else ends up using .random() or .getrandbits()).
msg268196 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2016-06-11 07:56
I don't want to start another huge thread on python-dev unless really necessary. What should happen to random.SystemRandom?

(1) nothing, it stays as it is, and if ``secrets`` needs better, it can subclass it;

(2) it changes to use ``os.getrandom``, and then ``secrets`` can continue to expose ``SystemRandom``.

If it comes to a vote, I'm +1 on option 2.
msg268247 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-11 18:54
Based on the documentation of random.SysRandom I think it comes down to #1. Perhaps the secrets documentation simply shouldn't mention the random module beyond how it's different? Then any use of the random module by the secrets module is an implementation detail that people can never rely on because we never said anything in the docs about it and clearly say "we do whatever is necessary to get you the most random bytes possible, backwards-compat be damned in how we pull it off".
msg268277 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2016-06-11 21:24
I think it's clear Guido would say "#1".  The thrust of all his comments to date is that it was a mistake to change the semantics of os.urandom() on Linux (and one other platform? don't really care), and that in 3.6+ only `secrets` should _try_ to supply possibly-blocking behavior on Linux.

Indeed, even in 3.6+ he doesn't want to expose the new-ish getrandom() directly.  Today he said:

"""
So what should the secrets module use? Let's make that part an extension module.
"""

So he feels strongly enough about this as to recommend a teensy private C extension module for secrets.py alone to get at the platform getrandom() (if any - of course it needs to fall back to os.urandom() on other platforms).

"""
The main thing that I want to avoid is that people start cargo-culting whatever the secrets module uses rather than just using the secrets module. Having it redundantly available as os.getrandom() is just begging for people to show off how much they know about writing secure code. 
"""

Whenever he starts taking that tone, his mind is made up for good ;-)
msg268278 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-11 21:24
It should be mentioned that Guido really doesn't like the idea of os.getrandom() as it exposes stuff where we would rather simply tell people to use the secrets module.
msg268287 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2016-06-11 21:58
I'm interested in implementing this for 3.6, but I'm not sure exactly what the changes are that were decided upon. Is the following accurate?

1. Nothing changes in os (all the necessary changes were made in 3.5.2 already).
2. On Linux 3.17+ only, we add a new extension module called _secrets that provides a wrapper around the getrandom() system call and nothing else. I'll have to figure out whether to make the extension module just not exist all on non-Linux OSs (are there any existing extension modules that only exist on one OS?), or whether to make it empty on non-Linux. _secrets.getrandom() will look like def getrandom(buflen: int, flags: int) -> bytes.
3. We change secrets.SystemRandom so that on Linux 3.17+ only, it subclasses random.SystemRandom to use _secrets.getrandom() instead of os.urandom(). We will not pass in any flags to getrandom() (so we read from /dev/urandom and block until there is sufficient entropy). To make the subclassing easier, we may want to change random.SystemRandom to have a private method for calling os.urandom. (Currently, both of its methods call os.urandom directly.)
4. Similarly, we change secrets.token_bytes to use _secrets.getrandom(). Other functions in secrets.py won't need to change since they use secrets.SystemRandom or secrets.token_bytes.
5. _secrets and _secrets.getrandom will remain private and undocumented.
msg268455 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-06-13 16:26
Setting this to "deferred blocker" priority to note that it needs to be resolved prior to feature code cutoff for 3.6.0, which is at 3.6.0b1 currently planned for 2016-09-07.
msg274709 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-09-07 02:31
PEP 524 has been implemented for 3.6b1 in #27776, so os.urandom() itself will now do the right thing for cryptographic use cases on Linux.
History
Date User Action Args
2022-04-11 14:58:32adminsetgithub: 71475
2016-09-07 02:31:27ncoghlansetstatus: open -> closed

nosy: + ncoghlan
messages: + msg274709

resolution: out of date
2016-06-16 13:25:57dstufftsetnosy: - dstufft
2016-06-13 16:26:06ned.deilysetpriority: release blocker -> deferred blocker

messages: + msg268455
2016-06-11 21:58:40JelleZijlstrasetnosy: + JelleZijlstra
messages: + msg268287
2016-06-11 21:24:35brett.cannonsetmessages: + msg268278
2016-06-11 21:24:01tim.peterssetmessages: + msg268277
2016-06-11 18:54:27brett.cannonsetmessages: + msg268247
2016-06-11 13:31:46Lukasasetnosy: + Lukasa
2016-06-11 07:56:05steven.dapranosetmessages: + msg268196
2016-06-10 20:20:55brett.cannonsetnosy: + steven.daprano
2016-06-10 20:15:49tim.peterssetnosy: + tim.peters
messages: + msg268152
2016-06-10 19:58:59dstufftsetmessages: + msg268149
2016-06-10 19:56:39dstufftcreate