This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author lemburg
Recipients Colm Buckley, Lukasa, Theodore Tso, alex, christian.heimes, doko, dstufft, larry, lemburg, martin.panter, matejcik, ned.deily, python-dev, rhettinger, skrah, thomas-petazzoni, vstinner, ztane
Date 2016-06-08.11:34:41
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <575802CA.7030207@egenix.com>
In-reply-to <1465381438.51.0.0280297469091.issue26839@psf.upfronthosting.co.za>
Content
Even though it may sound like a minor problem that os.urandom()
blocks on startup, I think the problem is getting more and more
something to consider given how systems are used nowadays.

Today, we no longer have the case where you keep a system up
and running for multiple years as we had in the past. VM,
containers and other virtualizations are spun up and down at
a high rate, so the boot cycle becomes more and more important.

FreeBSD, for example, is also concerned about the blocking issue
they have in their implementation:

https://wiki.freebsd.org/201308DevSummit/Security/DevRandom

and they are trying to resolve this by making sure to add as
much entropy they can find very early on in the process.

Now, most applications you run early on in the boot process
are not going to be applications that need crypto random
numbers and this is where I think the problem originates.

We've been telling everybody to use os.urandom() for seeding,
and so everyone uses it, including many many applications that
don't even require crypto random seeding.

The random module is the perfect example.

Essentially, we'd need to educate people that there's a difference
in requesting crypto random data and pseudo random data.

While we can fix the the cases in the stdlib and
the interpreter that don't need crypto random data to use
other means of seeding (e.g. reading straight from /dev/urandom
on Linux or gathering other data to mix into a seed), existing
applications out there will continue to use os.urandom() for
things that don't need crypto random numbers - after all, we told
them to use it.

Some of these will eventually be hit by the blocking problem,
even for applications such as Monte Carlo simulations that
don't need crypto random and should thus not have to wait for
some entropy pool to get initialized.

Now, applications that do need crypto random data should be
able to request this from Python via the stdlib and os.urandom()
may sound like a good basis, but since this is designed as
interface to /dev/urandom, it doesn't block on Linux, so
not such a good choice.

Using /dev/random probably doesn't work either, because this can
block unexpected even after initialization.

IMO, the best way forward and to educate application writers
about the problems is to introduce a two new APIs in 3.6:

os.cyptorandom() for getting OS crypto random data
os.pseudorandom() for getting OS pseudo random data

Crypto applications will then clearly know that
os.cryptorandom() is the right choice for them and
everyone else can use os.pseudorandom().

The APIs would on Linux and other platforms then use getrandom()
with appropriate default settings, i.e. blocking or raising
for os.cryptorandom() and non-blocking, non-raising for
os.pseudorandom().

As for the solving the current issue, we will have to
give people some way to get at non-blocking pseudo random data,
if they need it early in the boot process. With the
proposed change, this is still possible via reading
/dev/urandom directly on Linux, so not everything is
lost.

BTW: Wikipedia has a good overview of how the different
implementations of /dev/random work across platforms:

https://en.wikipedia.org/wiki//dev/random
History
Date User Action Args
2016-06-08 11:34:42lemburgsetrecipients: + lemburg, rhettinger, doko, vstinner, larry, christian.heimes, matejcik, ned.deily, alex, skrah, python-dev, martin.panter, ztane, dstufft, Lukasa, thomas-petazzoni, Colm Buckley, Theodore Tso
2016-06-08 11:34:42lemburglinkissue26839 messages
2016-06-08 11:34:41lemburgcreate