This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Theodore Tso
Recipients Colm Buckley, Lukasa, Theodore Tso, alex, doko, dstufft, larry, lemburg, martin.panter, matejcik, ned.deily, python-dev, rhettinger, skrah, thomas-petazzoni, vstinner, ztane
Date 2016-06-07.20:27:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1465331223.64.0.269660807987.issue26839@psf.upfronthosting.co.za>
In-reply-to
Content
Hi.   Colm alerted me to this bug, so I thought I would chime in as the author of Linux's getrandom(2) function.

First of all, if you are OK with reading from /dev/urandom, then you might as well use getrandom's GRND_NONBLOCK flag.  They are logically equivalent.

Secondly, when I decided to add this behavior to getrandom(2), it was because people were really worried that people would be using /dev/urandom for security-critical things (e.g., initializing ssh host session keys, when they'd _really_ rather not the NSA have be able to trivally pwn the server) before it had been completely initialized.   (And if it is not completely initialized, it would be trivially and embarassingly easy.  See https://factorable.net/weakkeys12.extended.pdf for an example of where this was rather disastrous.)

Why didn't I make /dev/urandom blocking?  Because a lot of people would whine and complain.   But getrandom(2) was a new interface, and so this was something I could do.   Now, before I decided to do this, I did do some benchmarks, and pre-systemd in practice on real hardware (e.g., x86 servers and laptops), I observed that you would actually see a message indicating that we had gathered 128 bits of entropy long before the root file system had been mounted.    With systemd, I observed that udevd was trying to read from /dev/urandom when we had only gathered an estimated 7 bits of entropy --- but I devoutly hoped that udevd wasn't doing anything super security critical, and trying to get the systemd people to change what they are doing is mostly like trying to teach a pig to sing, so I let it be.    However, in practice within a single digit number of seconds, the kernel printk indicating that random driver had considered itself initialized came quickly enough that I figured it would be safe to do.

If people are claiming that they are seeing cases where it takes over 90 seconds for the random number generator to initialize itself, please contact me directly; I'd love to know more, because that's input I would very much like to have.

However, at the end of the day, on certain hardware, if you don't have a source of initial entropy because the system doesn't have enough real hardware with real sources of entropy --- or if you don't trust your friendly cloud provider to provide you with some entropy from the hypervisor's entropy pool via virtio-random --- you can either (a) decide to pretend you are secure, when you really aren't, (b) wait, or (c) decide that you don't *really* need a secure source of randomness because you're really just initializing a hash for some associative array, and in fact srandom(time(0)) would have been fine, and you were using getrandom(2) or /dev/urandom just because you wanted to feel like one of the cool kids.

That being said, I do know of one potential issue which is if you happening to be using Microsoft Azure, the way the virtualized interrupt works, we weren't actually getting any entropy, and this was something I didn't discover until someone sent me a patch.  I have a patch[1] queued up in the random.git tree for the next kernel merge window to address that issue for Microsoft Azure servers. 

[1] http://git.kernel.org/cgit/linux/kernel/git/tytso/random.git/commit/?h=dev&id=8748971b4f5e322236154981827bf43dec4dc470

On a Google Compute Engine (GCE) system, I just did a quick test, and the "random: non-blocking pool initialized" message appears 5.64 seconds after the system is booted.  The changes I have queued up in random.git should reduce that to under a second.

All of this is neither here nor there, though.  The big question is *what* does Python expect to do with the randomness.  If you are just using it for computational simulation, you can do whatever you want.   If you are using it to create long-lived secrets that are intended to be secure against the depredations of a Nation-State's intelligence service, and you are on a system which really has almost no entropy available to be collected, then falling back to reading from /dev/urandom or using GRND_NONBLOCOK is going to be the equivalent of saying La-La-La-La-La-Nobody-Knows-How-Secure-I-Am while keeping your ears plugged.    (Now, if you are on an Intel system with RDRAND, and you trust Intel not to have given a back door to the NSA, you probably are safe, because we do actually mix in RDRAND.  On the other hand, if you are using some crappy ARM SOC for some Internet of Things device, and are firing up Python right after the system boots for the first time, and creating long-lived RSA private keys within milliseconds after the system is first booted --- please tell me so, I can avoid your product like the Plague.  :-)
History
Date User Action Args
2016-06-07 20:27:03Theodore Tsosetrecipients: + Theodore Tso, lemburg, rhettinger, doko, vstinner, larry, matejcik, ned.deily, alex, skrah, python-dev, martin.panter, ztane, dstufft, Lukasa, thomas-petazzoni, Colm Buckley
2016-06-07 20:27:03Theodore Tsosetmessageid: <1465331223.64.0.269660807987.issue26839@psf.upfronthosting.co.za>
2016-06-07 20:27:03Theodore Tsolinkissue26839 messages
2016-06-07 20:27:02Theodore Tsocreate