This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author larry
Recipients Colm Buckley, Lukasa, Theodore Tso, alex, christian.heimes, doko, dstufft, larry, lemburg, martin.panter, matejcik, ned.deily, python-dev, rhettinger, skrah, thomas-petazzoni, vstinner, ztane
Date 2016-06-08.08:04:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1465373073.53.0.0797074593352.issue26839@psf.upfronthosting.co.za>
In-reply-to
Content
I've revised my thinking on the subject.

First, my previous statements stand: Python startup must not block.  "import random" must not block.

Now I'm thinking that "os.urandom()" must not block, too.  Here's my reasoning.

--

If you read #25003, it's clear that /dev/urandom is a well-known UNIX facility with well-known, predictable behavior.  One behavior that I'll draw particular attention to now: it will never block.  If the system is low on entropy, /dev/urandom will produce lower-quality random bits.

Again I repeat myself: this is the *expected* behavior.  It is so completely the expected behavior, that today's special celebrity guest on the issue, Mr. Theodore Ts'o himself, added /dev/random specifically so it would be permitted to block when the system was low on entropy.  He *did not change* the behavior of /dev/urandom.  He added a *new device*.

A well-informed engineer would see "os.urandom()" and predict (correctly) that Python has provided a thin layer over /dev/urandom.  Thus os.urandom() should provide the same well-known, predictable behavior as /dev/urandom.

It's fine to enhance os.urandom().  For example, it's fine to provide higher-quality bits where available.  It's fine to provide the function on Windows which doesn't have a /dev/urandom object.

What is *not* fine is to degrade its behavior.  /dev/urandom is known to never, ever block.  This is a *feature*.  os.urandom(), therefore, must also never, ever block.

Yes, this means that on these cloud instances with no entropy (yet), os.urandom() may return these low-quality random bits.  Just like /dev/urandom does.

If I understand the APIs correctly, I'm fine with os.urandom() calling getrandom(,,GRND_RANDOM|GRND_NONBLOCK).  If that fails with EAGAIN it should fall back to reading from /dev/urandom, or getrandom(,,GRND_NONBLOCK) if that makes sense.  (IDK if that's Linux-specific; if it is I suppose /dev/urandom is the more cross-platform way to go.)

--

If this is seen as the end of the world by the crypto guys in the thread, let me say that I'm willing to consider adding a new function in 3.5.2.  I would propose it be spelled "os.getrandom(n, block=True)".  Crypto code could use this function if available, and fall back to os.urandom() where it was not.  This means you're covered: in 3.5.0 and 3.5.1 you use os.urandom(), and in 3.5.2+ you use os.getrandom(), and in both circumstances you'll block if there's insufficient entropy.

--

p.s. Colm Buckley: you notice how dstufft's patch got a "review" link, and none of the patches you posted got one?  That's because his is based on the current 3.5 repo and yours aren't.  This "review" link is very useful in reading your patches.  Please in the future try to base your patches against 3.5 trunk.  It's easy:
% hg clone https://hg.python.org/cpython/ 
% cd cpython
% hg up -r 3.5
(do your work here)
% hg diff > patchfile
(upload patchfile)
History
Date User Action Args
2016-06-08 08:04:33larrysetrecipients: + larry, lemburg, rhettinger, doko, vstinner, christian.heimes, matejcik, ned.deily, alex, skrah, python-dev, martin.panter, ztane, dstufft, Lukasa, thomas-petazzoni, Colm Buckley, Theodore Tso
2016-06-08 08:04:33larrysetmessageid: <1465373073.53.0.0797074593352.issue26839@psf.upfronthosting.co.za>
2016-06-08 08:04:33larrylinkissue26839 messages
2016-06-08 08:04:32larrycreate