This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Always use getrandom() in os.random() on Linux and add block=False parameter to os.urandom()
Type: enhancement Stage:
Components: Versions: Python 3.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Lukasa, Theodore Tso, larry, lemburg, martin.panter, ncoghlan, vstinner
Priority: normal Keywords: patch

Created on 2016-06-08 09:19 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
urandom_block.patch vstinner, 2016-06-08 09:19 review
Messages (63)
msg267814 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 09:19
Follow-up of the issue #26839: I propose to add a "block" keyword-only optional parameter to os.urandom(). I chose to make the parameter a keyword-only to avoid suprises and backward incompatible change and because I want to make this change in Python 3.5.2.

The attached patch:

* add block parameter to os.urandom()
* modify random.Random constructor to call os.urandom() with block=False
* modify _PyOS_URandom() to add a block parameter
* modify dev_urandom_noraise() to not block

The block parameter currently only changes the behaviour on Linux when the getrandom() syscall is used and the urandom entropy pool is not initialized.

With the change, os.urandom() blocks by default, but Python startup and "import random" doesn't block even if the urandom entropy pool is not initialized yet.
msg267821 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 09:43
Larry Hastings (msg267818, on the issue #26839):
> What is the default value of the "block" parameter?

False

> If called with block=False on FreeBSD, where /dev/urandom may block sometimes, what does the function do?

The function blocks.

In which case FreeBSD urandom can block?

Is there a way to read from /dev/urandom without blocking? If not, _PyOS_URandom() may use arc4random() on FreeBSD if block is false. I suggest to enhance _PyOS_URandom() after the Python 3.5.2 release.

My patch is a quick fix for Python 3.5.2, it can be enhanced later.

I chose to only modify the behaviour on Linux since issues #26839 and #25420 were only reported on Linux.
msg267824 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 09:45
This issue is the opposite of the issue #27250 "Add os.urandom_block()".
msg267828 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 10:03
> Is there a way to read from /dev/urandom without blocking? If not, _PyOS_URandom() may use arc4random() on FreeBSD if block is false. I suggest to enhance _PyOS_URandom() after the Python 3.5.2 release.

See the issue #22542 "Use arc4random under OpenBSD for os.urandom() if /dev/urandom is not present".

It looks like arc4random() is now initialized with getentropy(KEYSZ+IVSZ) (32+8 = 40 bytes) on OpenBSD:
http://bxr.su/OpenBSD/lib/libc/crypt/arc4random.c

getentropy() is non-blocking. By the way, os.urandom() is now implemented with getentropy() on OpenBSD.
msg267829 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 10:06
I'm pretty sure that getentropy on OpenBSD will block until the pool is initialized.
msg267867 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 16:18
Ah, there is maybe a solution for BSD?

msg267810: "it looks like we can use sysctl to fetch the seed state from kern.random.sys.seeded"
msg267868 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 16:23
I changed the title to make my intent more explicit: the goal is to make os.urandom() more secure on Linux by always using getrandom(). But always using getrandom() in os.urandom() requires a non-blocking os.urandom() to start Python: see #25420 (import random) and #26839 (hash secret) for practical issues.

I removed Python 3.5 from versions since Larry explicitly asked to not add a new parameter to os.urandom() in Python 3.5: msg267721.
msg267878 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 18:40
First, DO NOT merge this change into 3.5.2 without my explicit permission.

Second, unless you can guarantee you support blocking / non-blocking behavior on all platforms, this is a bad move.
msg267879 - (view) Author: Theodore Tso (Theodore Tso) Date: 2016-06-08 19:02
Larry, at least on FreeBSD, it sounds like the implementation could just the kern.random.sys.seeded sysctl, and return <something>.  (Note: what is the proposed behaviour if the PRNG is not seeded?  Return Null?)

As far as OpenBSD is concerned, it's true that it's getentropy(2) never blocks.  But that's because OpenBSD assumes that the bootloader can read a seeded entropy file from the previous boot session, and that the CD-ROM installer will be able to gather enough entropy to save a entropy file from the beginning of the installation.    So if you don't have worry about booting your OS on an ARC Internet of Things device, you can afford to make certain simplifying assumptions.

Could Linux on x86 do something similar (read the entropy file in the bootloader)?  Sure, the design isn't difficult.  If someone can fund the headcount to do the work, I'd be happy to help supervise the GSOC intern or work with some engineer at some other company who is interested in getting a change like this upstream.  But there will still be cases where getrandom(2) could block, simply because I can't control all of the hardware platforms where Linux might be ported to.   The real problem is that since on real hardware platforms it's only a problem in very early boot, it's hard to make a business case to invest in solving this problem.
msg267882 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 19:54
Larry Hastings added the comment:
>
> First, DO NOT merge this change into 3.5.2 without my explicit permission.
>

I explicitly removed Python 3.5 from this issue to follow your request. See
my previous comment :-)
msg267883 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 20:00
> getentropy() is non-blocking. By the way, os.urandom() is now implemented with getentropy() on OpenBSD.

I quote issue #25003:

> You can not substitute getentropy() for getrandom(), if you need randomness then entropy does not suffice.
msg267917 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-08 22:29
> Second, unless you can guarantee you support blocking / non-blocking behavior on all platforms, this is a bad move.

Well, there are two options:

* Detect when os.urandom() is going to block, and falls back to weaker entropy (Linux: /dev/urandom in non-blocking mode, others: getpid+time?).
* Never use os.urandom() to seed random.Random when the randim module is imported: I suggest to discuss this option in the issue #27272

I would prefer to collect enough technical information before taking a decision (ensure that it's not possible to get a portable function to check if /dev/urandom is going to block.)
msg267922 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 22:54
This is why I'm unwilling to accept this change in 3.5.2.  This sort of "we think it's right" behavior is only appropriate in a point release like 3.6.

I think odds are better than even that I'm going to get an os.getrandom(n, block=True) call in 3.5.2.  It solves problems, and its behavior is predictable on a platform-by-platform basis.  It also won't change the behavior of any reasonable existing code.  (Yes, someone could say "if hasattr(os, 'getrandom'): detonate_the_device()" but that's unreasonable.)

If os.getrandom(n, block=True) goes in 3.5 and 3.6, this makes adding a block parameter to os.urandom() less interesting.

Also I disagree with adding the block parameter to os.urandom() in the first place.  os.urandom() should behave like the /dev/urandom device on the local OS.  Whether or not /dev/urandom blocks is system-dependent behavior. On Linux, it's guaranteed to never block; on FreeBSD it is permitted to block under certain conditions.  At the point that you say "but on Linux we're implementing os.urandom() using the getrandom() system call, which has the ability to block if you want", you are straying quite far from the behavior of os.urandom().

Functions in the os module are intended to provide a thin shell around the equivalent local OS function, and should behave predictably like that local function.  os.stat() should behave like the local stat(); os.utime() should behave like the local utime(); and os.urandom() should behave like the local /dev/urandom.  It's okay to add functionality for free, but it's not okay to degrade behavior (like "it might block forever").

Since the behavior of /dev/urandom already varies so widely between different platforms, I think it adds unhelpful complexity and confusion to add the block= parameter.  I think we're much better off with a new function providing the local system call: os.getrandom(), if available, and perhaps os.getentropy(), if available.  Then we preserve the "behaves predictably like the local system call" approach of the os module.
msg267923 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 22:56
It would still be a thin shell around what the OS provides, it'd just be a thin wrapper around getrandom() instead of around /dev/urandom specifically-- but accessing the same content.
msg267924 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 22:58
I think a new function is a bad idea TBH, os.urandom provides 99% of the right interface on Linux, and 100% of the right interface on other OSes and there is a lot of code out there using os.urandom for it's security properties, and a lot of people educating users to use os.urandom for security sensitive tasks.

All of a sudden going, "wait! You actually want Y" is not going to make anything more clear, especially since the difference in behavior only matters in an edge case.
msg267925 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 23:14
os.getrandom() would be a thin shell around what the local OS provides, and has the advantage of behaving in a predictable manner.  If you provide block=, its implementation and semantics will vary wildly between platforms.

* On Linux, block=False should be the default.  block=True means it will use getrandom(), and block=False means it will use /dev/urandom except possibly getrandom().

* On FreeBSD, block=True will be the default, and block=False will either be ignored, or return an empty string (or throw an exception, depending on what would be a better error-handling mechanism, IDK).

* On OS X, block=False would be the default, and block=True would be ignored, because the interface doesn't permit blocking to wait for additional entropy and generating higher-quality random bits.

So, yes, the os module should *expose* the behavior of the underlying OS, rather than trying to paper over it.

> there is a lot of code out there using os.urandom for it's security properties

This is exactly why we should not change the behavior of os.urandom().  os.urandom() must not block on Linux.  So defaulting to block=True on Linux is a non-starter.

The argument "if we add a block parameter, then users can use it and benefit", is equivalent to saying "if we add a new function os.getrandom(), then users can use it and benefit".  The user needs to be educated, and we can do that with either approach.

However, if users want to write backwards-compatible code, it's a lot easier to detect the presence or lack of a new function (hasattr(os, "getrandom")) than to detect the presence or lack of a new parameter to an existing function (which, if we're lucky, we could do with inspect.signature, and if we're unlucky involves a try/except block).
msg267926 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 23:14
I also contest the idea that os functions must only be thin shells around system provided calls (even though I don't think that using getrandom instead of reading from /dev/urandom violates that assertion).

There are a number of functions in os.py that add additional logic ontop of the system calls, like:

* os.makedirs
* os.removedirs
* os.renames
* os.walk
* os.popen (which actually imports and uses the entire subprocess module)
* os.scandir

That's just from a quick scan of the pure Python os.py file.
msg267927 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 23:16
> > there is a lot of code out there using os.urandom for it's security properties
>
> This is exactly why we should not change the behavior of os.urandom().  os.urandom() must not block on Linux.  So defaulting to block=True on Linux is a non-starter.

This statement doesn't make any sense to me... you're asserting that because a lot of people are using os.urandom assuming it's going to give them cryptographically secure random numbers... we shouldn't change the implementation of this function to assure that they are going to get cryptographically secure random numbers?
msg267930 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 23:32
> There are a number of functions in os.py that add additional logic ontop of the system calls, like:

All the functions you name don't have POSIX equivalents, except popen and scandir.

popen provides a lot of functionality around popen (which internally does fork()/exec()).  But fundamentally it behaves like popen, and its behavior is predictable across platforms.

scandir() says right in the documentation: it's implemented using opendir() and readdir().  This makes the behavior predictable across platforms.

You might have also cited os.utime(), which hides byzantine complexity internally and may be implemented with one of about a half-dozen local calls.  But is behavior is predictable across platforms.

/dev/urandom behaves very differently on different platforms, to an extent that CPython can't reasonably be expected to hide the cross-platform behavior and behave in an outwardly predictable manner.  On Linux, by default it should never block, and we could theoretically provide additional functionality to let it block and guarantee high-quality random bits.  On OS X, we don't have any option: it will never block, and it will never provide high-quality random bits.  On FreeBSD, we don't have any option: it could locally block, and if the user would prefer to not block and get lower-quality random numbers as a result, sorry they're out of luck.

It's not reasonable for os.urandom() to hide this complexity.  It is better that os.urandom() is a thin layer around /dev/urandom, and behaves predictably like /dev/urandom on the local computer.  The block= parameter confuses the matter greatly, which is why I oppose it.


> This statement doesn't make any sense to me... you're asserting that because a lot of people are using os.urandom assuming it's going to give them cryptographically secure random numbers... we shouldn't change the implementation of this function to assure that they are going to get cryptographically secure random numbers?

You misunderstand me.  I assert that people are calling os.urandom() because they expect it to behave like /dev/urandom on their machine.  I make no assertion what they expect to happen as a result.  What is returned by os.urandom() is platform-dependent.

If you read the documentation for os.urandom(), it specifically does *not* guarantee the result is suitable for cryptography.  The one-line summary seems to suggest that it does, but then in the very next paragraph it clarifies the situation:

"This function returns random bytes from an OS-specific randomness source. The returned data should be unpredictable enough for cryptographic applications, though its exact quality depends on the OS implementation."

I *am* asserting that /dev/urandom is guaranteed to not block on Linux, therefore os.urandom() should never block on Linux.
msg267931 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 23:43
Let me make one more thing clear.  I'm willing for os.urandom() to try to use getrandom(GRND_NOBLOCK) (or whatever the flag is called).  This will not block unless the entropy pool is low--which almost never happens.  So 99.9999% of the time, os.urandom() should return lovely high-quality, cryptographically-safe random numbers.

Really this entire debacle is an argument about this funny edge cases, like "you create a new VM and run python3 as PID 0 and your bad sysadmins don't manage the entropy pool and the entropy pool never fills".

What should the code do in that situation?  Is it acceptable to use low-quality bits from /dev/urandom?  Or do they need cryptographically-strong random numbers?  I quote the Zen: "In the face of ambiguity, refuse the temptation to guess."  Thus, we shouldn't block on Linux, and we should behave predictably like /dev/urandom does on the local machine.

This leads me to one reason why I oppose block=.  It's hiding the complexity of the local system with even more complexity, and makes it hard to reason about what the code is doing.  The os module should behave in a predictable manner, as if it was a thin shell around the local system call.  Given that it's impossible to block on Linux and get higher-quality random bits, and it's impossible to *not* block on FreeBSD and get lower-quality random bits, adding block= to os.urandom() means its behavior becomes inobvious and hard to predict.
msg267934 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-08 23:51
> The block= parameter confuses the matter greatly, which is why I oppose it.

I don't like the block= parameter.

> I assert that people are calling os.urandom() because they expect it to behave like /dev/urandom on their machine.

I don't think most people calling os.urandom have any idea how /dev/urandom behaves on their machine nor do I think most people have any idea how /dev/urandom behaves on other people's machines. In fact, in the original thread there were long time developers stating factually wrong information about the properties of os.urandom. That's not a mark against them, the behavior of random is an esoteric and nuanced thing and there is a lot of misinformation out there about the properties and behavior of it on various systems.


> If you read the documentation for os.urandom(), it specifically does *not* guarantee the result is suitable for cryptography.

Sure, but the cases where os.urandom doesn't provide results suitable for cryptography are:

* Systems where you have a random number generator that is completely broken, in which case there's basically no help for you and you're screwed 10 times over before you even execute Python.
* Linux, in the window prior to the pool being initialized.

That statement is an obvious hedge against the first case, not an allowance for the second case in situations where we don't have to continue to use a worse interface for the same underlying functionality.

If chdir on Linux would, in 0.01% of cases, cause a segfault, but 99.9% of cases work as you'd expect and a new, chdir2 function was added that still changed the directory, but did so in 100% of cases, would you be opposed to changing os.chdir to use chdir2, even though it's no longer a thin wrapper around chdir or would you demand that all developers switch to os.chdir2? Especially if this behavior only really occurred on Linux.
msg267935 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-08 23:59
Wait a minute.  If I read the code correctly, currently os.urandom() is implemented strictly using getrandom() *on all platforms*.  And if block=false, it... returns an empty buffer?  Am I reading that right?

And what does it do on platforms that don't have a getrandom() call?  Does os.urandom() literally always raise an exception on OS X or FreeBSD?

This is so crazy.  Surely I'm misunderstanding the code.  What am I missing?
msg267937 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 00:03
> I don't think most people calling os.urandom have any idea how /dev/urandom behaves on their machine nor do I think most people have any idea how /dev/urandom behaves on other people's machines.

Here I invoke the "consenting adults" rule.  In Python development, we never say "The user doesn't know what they're doing here, so we need to do the right thing for them."  You must treat Python programmers as adults and assume they know what they're doing.

If the user is calling os.urandom(), which is documented as behaving like /dev/urandom, then it must behave like /dev/urandom.  We can optionally make it behave better than /dev/urandom, but not at the cost of unpredictable complexity, and not at the cost of degraded performance (unbounded blocking).
msg267938 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-06-09 00:04
Transferring a comment I previously made on #27250:

One request I'd make in these discussions is that we avoid using the term "block" - it makes people think of the /dev/random behaviour (i.e. blocking intermittently and unhelpfully), rather than the usually-desired "wait for sufficient entropy on system startup" behaviour. (The only documented case where that behaviour clearly *isn't* desired is for people writing startup scripts on Linux that may run before the system entropy pool is initialised, since doing that has been shown to deadlock the system until the systemd script watchdog times out)

I'd also request that we keep in mind that any Linux user always remains free to write the 3-line utility function:

    def read_urandom(num_bytes):
        with open('/dev/urandom', 'rb') as urandom:
            return urandom.read(num_bytes)

If they want to get precisely the Linux /dev/urandom semantics, and not a Python level abstraction that provides the same kinds of assurances offered by other *nix platforms and by the Windows crypto APIs. While "os" originated as a relatively thin wrapper around POSIX APIs, that's far from universally true these days, especially given the introduction of things like implicit Unicode handling, implicit EINTR handling, os.scandir, dir_fd handling, and more.

I'd also *STRONGLY* request that we avoid adding any new APIs in relation to this that mean "Use os.urandom" is no longer the preferred option to obtain cryptographically strong random numbers in Python. Any such APIs can't be used in single source Python 2/3 code, they invalidate existing third party documentation like https://cryptography.io/en/latest/random-numbers/ and they invalidate answers on Q&A sites like http://stackoverflow.com/questions/20936993/how-can-i-create-a-random-number-that-is-cryptographically-secure-in-python
msg267940 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-09 00:14
Python should strive do the right thing, and the right thing for something as security sensitive as accessing a CSPRNG is to use the interface that most strongly promises to in fact, give cryptographically secure random numbers. Obsessing over the purity of matching /dev/urandom because the function happens to be called os.urandom is only making Python more dangerous to use.

This is exactly the kind of change like making urllib validate HTTPS by default, it doesn't matter what you document something as behaving, what matters is how people use it and what the expectations are of the average user. The nice thing about this change, is the downside is massively smaller than that urllib change, because it's basically not going to negatively affect the vast vast bulk of people. 

I think this is doubly so since to get the behavior you desire on Linux is trivial to do regardless of what os.urandom does, which is using the code snippet that Nick pasted.

Larry, I'm sorry but I think you're just flat wrong here and I don't know what else to say about it. Since you're the RM for 3.5 and you've made it clear you're against the behavior I'm advocating for, I'm going to respect your decision on that and I'm not going to pursue getting it into 3.5, however I am going to pursue getting it into 3.6.
msg267941 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-06-09 00:15
As a process comment: I agree with what Victor wrote in http://haypo-notes.readthedocs.io/pep_random.html#status-of-python-3-5-2, when he suggests that we leave 3.5.2 as is for now (i.e. reverted to be consistent with 2.7 and 3.4, which favours a risk of subtle security failures in some situations over a certainty of Python failing to start in those same situations), and only reconsider what to do for 3.5.x and 2.7.x after we have agreed on an acceptable solution for 3.6.
msg267942 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 00:24
> I'd also *STRONGLY* request that we avoid adding any new APIs in relation to this that mean "Use os.urandom" is no longer the preferred option to obtain cryptographically strong random numbers in Python.

According to the documentation, os.urandom() does not actually *guarantee* cryptographically strong random numbers in Python.  Indeed, it has never guaranteed this; you can check the 2.6 documentation, it says the same thing.

There are debates among cryptographers about how strong the numbers are.  The paper cited by Mr. Ts'o shows that the numbers can be pretty awful under Linux when the entropy pool is empty.

The only way of fixing this this is by using a different API, which can block in an unbounded fashion.  It's simply unreasonable to change to this API by default, because on Linux /dev/urandom is guaranteed to never block.

I don't see it as an improvement to add a block= parameter to os.urandom(); the parameter behaves differently on different platforms (currently ignored on most platforms) and if implemented would make the underlying implementation and behavior far more difficult to predict and reason about.

Mr. Ts'o had exactly the same problem on Linux.  /dev/urandom was the preferred way of obtaining cryptographically strong random numbers.  But it didn't produce them in all cases, and in order to produce them he really had to block.  Instead of changing /dev/urandom so it would sometimes block, he added a new kernel API (getrandom()) which is permitted to block, and he basically left /dev/urandom alone.  I see Python as facing the same problem, and I think it should solve the problem in the same way: leave os.urandom() alone and add a new function getrandom() where possible.

I would support a new function in 3.6, something like random.get_cprng_bytes(n) (probably a bad name), which always returns cryptographically strong PRNG bits and is permitted to block in an unbounded fashion.  The complication here is I'm not sure we can provide it on every platform; the quality of bits available from /dev/random on OS X is apparently highly debatable.  (OTOH, what aspect of cryptography is not highly, endlessly, debatable? *sigh*)
msg267943 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 00:29
> As a process comment: I agree with what Victor wrote in http://haypo-notes.readthedocs.io/pep_random.html#status-of-python-3-5-2, when he suggests that we leave 3.5.2 as is for now [...]

I agree in principle.  Certainly we all agree it'd be inappropriate to add the block parameter to 3.5.2.

I only allowed the possibility of adding os.getrandom() in 3.5.2 to try and placate the authors of cryptography libraries.  They could use os.getrandom() if it was available, and fall back to os.urandom() if not.  Since they want the blocking behavior, that means they'd block in 3.5.2+ (they'd call getrandom), and they'd also block in 3.5.[0-1] (they'd call os.urandom() which blocks on Linux in those versions).
msg267944 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 00:34
> Larry, I'm sorry but I think you're just flat wrong here and I don't know what else to say about it.

Yeah, I think maybe asking Guido to make a ruling would be the right thing here.  Again, I cite #25003: he implicitly ruled there that it's not permissible for os.urandom() to block on Solaris, so I think he agrees with me.  But if he made a ruling naturally I'd acquiesce to it regardless of what he decided.
msg267945 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-09 00:35
> Again, I cite #25003: he implicitly ruled there that it's not permissible for os.urandom() to block on Solaris, so I think he agrees with me.

To be clear, that's an entirely different kind of blocking. That made os.urandom on Solaris behave similarly to /dev/random, where it would randomly block for all users of that API regularly. I would have made the same ruling, this blocking is not that blocking.
msg267947 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 01:08
About the "add block=False parameter to os.urandom()" part of the issue title: I don't really care of the exact API. It can be a new function. The parameter can be named differently. The idea is to give access to users to Linux non-blocking /dev/urandom, and do something helpful for users on platforms without non-blocing urandom.
msg267948 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 01:50
Victor: just to confirm, would "os.getrandom(n, block=True)" be okay with you?  Do you strongly prefer adding the "block" parameter to os.urandom(), or do you not care much?
msg267962 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 05:48
IHMO we only need two functions. os.urandom() is now documented as the
reference for security. So we need a new function. I proposed a new
urandom() parameter, but a new function without parameter would also be ok.

If os.getrandom() is directly a binding of Linux/Solaris getrandom(), I
will not use it since it would not be portable.
msg267963 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 05:51
AFAICT, the history is, Linux added /dev/urandom, then all the other UNIXes followed suit.  I've read a lot of man pages for /dev/urandom (or equivalent) on a lot of different platforms and most of them say "this was added to Linux, so we added our own version".

/dev/random and getrandom() seem to be playing out the same way.  Linux added getrandom() first, then Solaris added it.  My bet is most of the other UNIXes will add it over the next couple years.
msg267966 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-09 07:31
I dislike the “block” term for a different reason: it suggests raising EAGAIN (= BlockingIOError). But the proposal here is actually to generate data with low entropy.

In the long term, it sounds like two separate functions is the way to go. I would prefer os.urandom() to retain its 3.5.1 behaviour introduced by Issue 22181: wait for sufficient entropy. This would be consistent with other platforms. It can continue to be used by SystemRandom, secrets, etc. The documentation should be clarified to say that os.urandom() either returns data that is supposed to be “unpredictable enough for cryptographic applications”, blocks waiting for entropy, or raises an NotImplementedError if an OS API (e.g. /dev/urandom) is missing.

The other function, say os.urandom_even_with_low_entropy(), would work like the old os.urandom() on Linux. On platforms like Free BSD, I guess it could fall back to the os.urandom() that waits for entropy. “Import random” would be changed to try this function.

I wonder what we should do on Linux if /dev/urandom is unavailable and getrandom() would block. Probably raise NotImplementedError without waiting for entropy.
msg267968 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-06-09 07:41
Can we please stop using the term "low entropy" here. It is wrong and makes my head hurt. "Running out of entropy" is an urban myth. It's about the initialization state of Kernel's internal RNG. The Kernel blocks the syscall as long as it is not confident enough about its RNG. It wants to collect a decent amount of events from different sources until it considers its RNG properly seeded. Once the RNG is seeded, you can pull as many data from the RNG as you like.

It's really block_on_boot_until_kernel_rng_has_been_seeded.
msg267971 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-06-09 07:56
I propose to deprecate os.urandom() altogether due to all the
issues we've discussed on all those recent tickets.

See #27279 for details.

On 09.06.2016 02:04, Nick Coghlan wrote:
> I'd also *STRONGLY* request that we avoid adding any new APIs in relation to this that mean "Use os.urandom" is no longer the preferred option to obtain cryptographically strong random numbers in Python. Any such APIs can't be used in single source Python 2/3 code, they invalidate existing third party documentation like https://cryptography.io/en/latest/random-numbers/ and they invalidate answers on Q&A sites like http://stackoverflow.com/questions/20936993/how-can-i-create-a-random-number-that-is-cryptographically-secure-in-python

It's easy enough to write Python2/3 code to use the new
APIs for Python3, so I don't really buy that argument.

As for answers on SO: it's not a definite source for anything
anyway and life moves on, so eventually people will up the
new changes and update older answers to the new ones. The
deprecation notice will get people aware of the change.
msg267973 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 07:57
> I wonder what we should do on Linux if /dev/urandom is unavailable and getrandom() would block.

I don't think that happens.

getrandom() actually supports two flags.  The flag GRND_RANDOM tells it "behave like /dev/random".  If you don't pass in GRND_RANDOM, it behaves like "/dev/urandom".  So it's hard to imagine that you could have getrandom() and not have /dev/urandom.
msg267974 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 07:57
I played with select() on Linux on a VM:

* /dev/random: it works as expected
* /dev/urandom: the device is already seen as readable even before the urandom entropy pool is initialized. It is not really surprising since, yes, read() does not block in practice

To test Python before urandom is initialized, I used the init=/path/to/python trick in the boot loader.

By the way, I confirm that getrandom(GRND_NONBLOCK) fails with EAGAIN before urandom is initialized.

--

I'm now trying to get a non initialized /dev/urandom in a FreeBSD VM, but it seems harder... Even if single user mode, select() on /dev/urandom says that the device is ready and yes, it works to read a few bytes. Moreover, "sysctl kern.random.sys.seeded" returns 1, which confirms that urandom is already initialized.

If even in single mode urandom is already is initialized in a VM, I'm not sure that FreeBSD is really impacted by the issue #26839.
msg267976 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 08:02
> I dislike the “block” term for a different reason: it suggests raising EAGAIN (= BlockingIOError). But the proposal here is actually to generate data with low entropy.

Since os.urandom() is part of the os module, yes, I'm suggesting to raise BlockingIOError. The caller would be responsible to handle this error and make a choice. For example, random.py can reuse its fallback on time.time().
msg267979 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 08:08
Martin:
> I wonder what we should do on Linux if /dev/urandom is unavailable and getrandom() would block.

os.urandom(block=False) should raise BlockingIOError if getrandom(GRND_NONBLOCK) fails with EAGAIN and /dev/urandom is not available.

Larry:
> I don't think that happens. getrandom() actually supports two flags.  The flag GRND_RANDOM tells it "behave like /dev/random".  If you don't pass in GRND_RANDOM, it behaves like "/dev/urandom".  So it's hard to imagine that you could have getrandom() and not have /dev/urandom.

You can imagine a "badly configured" container or chroot where /dev or only /dev/urandom doesn't exist.

When I played with chroot, it was easy to forget /dev/urandom. Since Python uses getrandom() on Linux since Python 3.5, you can imagine that a Python 3.5 user may not notice the lack of /dev/urandom in the common case (urandom initialized), but start to get errors when the container runs before urandom is initialized.
msg267980 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 08:10
Fair enough.  But Theodore Ts'o said on the tracker: if you call getrandom() and don't pass in GRND_RANDOM, it's equivalent to /dev/urandom.  So, if getrandom is available, getrandom(flags=0) will always work and never block.
msg267981 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-06-09 08:11
I'm -1 on a block=True/False parameter. It makes the API more awkward and will make people move away from os.urandom() to a self-made RNG because they perceive os.urandom() as potential blocking.

Victor, you can use sysctl on all BSD and Linux to check if the RNG has been seeded. On Linux it is kernel.random.entropy_avail >= kernel.random.read_wakeup_threshold. On BSD it is kern.random.sys.seeded == 1.
msg267982 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-06-09 08:11
On 09.06.2016 09:57, STINNER Victor wrote:
> 
> STINNER Victor added the comment:
> 
> I played with select() on Linux on a VM:
> 
> * /dev/random: it works as expected
> * /dev/urandom: the device is already seen as readable even before the urandom entropy pool is initialized. It is not really surprising since, yes, read() does not block in practice
> 
> To test Python before urandom is initialized, I used the init=/path/to/python trick in the boot loader.

It's best to look at the code for this in Linux:

http://lxr.free-electrons.com/source/drivers/char/random.c

getrandom() is implemented directly on top of the two
devices:

http://lxr.free-electrons.com/source/drivers/char/random.c#L1601

> By the way, I confirm that getrandom(GRND_NONBLOCK) fails with EAGAIN before urandom is initialized.

Right, and that's good, since it's better to let the application
control what to do than to simply block.

Here's a good reading on getrandom():

https://lwn.net/Articles/606141/
msg267985 - (view) Author: Cory Benfield (Lukasa) * Date: 2016-06-09 08:14
> But Theodore Ts'o said on the tracker: if you call getrandom() and don't pass in GRND_RANDOM, it's equivalent to /dev/urandom.  So, if getrandom is available, getrandom(flags=0) will always work and never block.

Can we please try to be clear about what kind of blocking we mean? getrandom(flags=0) absolutely *can* block: that's what the original issue was all about. To ensure it *never* blocks you need to call getrandom(GRND_NONBLOCK): that's why the flag exists.

Put another way:

- getrandom(flags=GRND_RANDOM) == /dev/random
- getrandom(flags=GRND_NONBLOCK) == /dev/urandom on Linux
- getrandom(flags=0) == /dev/urandom everywhere but Linux
msg267987 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 08:16
Marc-Andre Lemburg:
> I propose to deprecate os.urandom() altogether due to all the issues we've discussed on all those recent tickets.

I'm sorry, but I don't understand the purpose of this change. Usually, when we deprecate something, it is in favor of a new better function. What do you propose?

I read that you proposed to expose getrandom() as os.getrandom(). It would be painful to write portable code if each OS provides its own RNG function.

Python has the habit of helping users by providing portables functions. Recent example: time.monotonic (PEP 418). Somehow related: non inheritable file descriptors by default (PEP 446) and retry system calls failing with EINTR (PEP 475). These changes aim to simplify the life of Python developers to reduce the subtle differences between each operating system.

To me, os.urandom() is well defined. The corner case of not initialized urandom is really a corner case which only occurs in "catastrophic" cases like ("badly configured") VM or embedded devices without hardware RNG (nor RTC).

When it's hard to write a reliable behaviour on all platforms, the simple solution was always to document the subtle differences between each platforms. I started to do with documenting getrandom() and the fallback on /dev/urandom for Linux:
https://docs.python.org/dev/library/os.html#os.urandom

--

If we cannot agree on a technical solution, a PEP is required.

But please give me some time to investigate the different technical solutions before trying to take a decision.

Right now, I'm investigating the options to keep the Python startup "secure" in the "urandom not initialized" case and keep os.urandom() "blocking".
msg267988 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 08:19
If you didn't read it yet, I suggest you to read my summary of the issue #26839 which contains a lot of information on RNG in the case of Python:
http://haypo-notes.readthedocs.io/pep_random.html

It can be used to write a real PEP.
msg267993 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 08:41
> Can we please try to be clear about what kind of blocking we mean? getrandom(flags=0) absolutely *can* block: that's what the original issue was all about. To ensure it *never* blocks you need to call getrandom(GRND_NONBLOCK): that's why the flag exists.

Thanks, I was actually confused on this issue.  I thought CPython was using getrandom(GRND_RANDOM) and that's why it was blocking.  But to be clear, you're right: 3.5.1 is calling getrandom(0) in all circumstances.  It never passes in GRND_RANDOM and it never passes in GRND_NOBLOCK.  And according to the manpage for getrandom(), getrandom(0) "blocks if the entropy pool has not yet been initialized".

What I don't understand is this, from the manpage for urandom:

> A read from the /dev/urandom device will not block waiting for more entropy.  If there is not sufficient entropy, a pseudorandom number generator is used to create the requested bytes.

If both sources are right, then /dev/urandom behaves quite differently from getrandom(0).

Imagine how confused I was when Theodore Ts'o said:

> First of all, if you are OK with reading from /dev/urandom, then you might as well use getrandom's GRND_NONBLOCK flag.  They are logically equivalent.

He wrote it.  But what he said there doesn't jibe with what the manpages say.  Those say that if you call getrandom(GRND_NONBLOCK) before the entropy pool has been initialized, it will return EAGAIN, but any time you read from /dev/urandom you will always get random data.

... the more I learn about this, the less I think I understand it.
msg267994 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 08:41
I also tested to call os.urandom() "early" in the Windows boot process in a Windows VM. Using a startup entry in the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run registry key, os.urandom() works immediatly (it is able to produce 16 bytes of entropy), whereas I didn't touch the mouse, keyboard, or anything. The VM is running with qemu without RNG device (I got one, I removed it).

"early" is not correct: it's late in fact, only when the desktop is opened. I should use GPEDIT.MSC to start a task before the desktop, but this command doesn't seem to be available on my Windows 8?
http://stackoverflow.com/questions/614766/run-a-script-on-windows-startup-without-a-user-logged-on
msg267996 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-06-09 08:49
On 09.06.2016 10:16, STINNER Victor wrote:
> 
> STINNER Victor added the comment:
> 
> Marc-Andre Lemburg:
>> I propose to deprecate os.urandom() altogether due to all the issues we've discussed on all those recent tickets.
> 
> I'm sorry, but I don't understand the purpose of this change. Usually, when we deprecate something, it is in favor of a new better function. What do you propose?

It's mainly to get people stop thinking that we can fix os.urandom()
in a way so that it works the same across platforms and to
draw some attention to the main proposal in #27279 :-)

I don't feel strong about the deprecation.

> I read that you proposed to expose getrandom() as os.getrandom(). It would be painful to write portable code if each OS provides its own RNG function.

This would be done in the two new functions, in Python and
based on whatever is available on the platform. See #27279.

Application writers should not have to worry about the
different mechanisms used on the different platforms.

> Python has the habit of helping users by providing portables functions. Recent example: time.monotonic (PEP 418). Somehow related: non inheritable file descriptors by default (PEP 446) and retry system calls failing with EINTR (PEP 475). These changes aim to simplify the life of Python developers to reduce the subtle differences between each operating system.

Exactly.

> To me, os.urandom() is well defined. The corner case of not initialized urandom is really a corner case which only occurs in "catastrophic" cases like ("badly configured") VM or embedded devices without hardware RNG (nor RTC).

It is a corner case, but one we'll see hit us more often going
forward, since booting up VMs/containers no longer is a rather
rare incident. It's being used as part of the architecture of
system nowadays.

> When it's hard to write a reliable behaviour on all platforms, the simple solution was always to document the subtle differences between each platforms. I started to do with documenting getrandom() and the fallback on /dev/urandom for Linux:
> https://docs.python.org/dev/library/os.html#os.urandom
> 
> --
> 
> If we cannot agree on a technical solution, a PEP is required.

Right. The topic has too many aspects to really address in a bug
tracker ticket. It's better to start writing these things down
in a PEP. I'm using #27279 as scratch pad for this.

> But please give me some time to investigate the different technical solutions before trying to take a decision.

Sure, Python 3.6 is still in the making...

> Right now, I'm investigating the options to keep the Python startup "secure" in the "urandom not initialized" case and keep os.urandom() "blocking".

The Python startup definitely has to be fixed, regardless of what
we do with os.urandom(). This means: getting hash randomization
and the random module initialized without blocking even when
the entropy pool is not yet initialized.

I guess the easiest way to solve this for the random module
is by making the initialization lazy (as was proposed on the
resp. ticket).
msg267997 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-06-09 09:06
On 2016-06-09 10:49, Marc-Andre Lemburg wrote:
> It is a corner case, but one we'll see hit us more often going
> forward, since booting up VMs/containers no longer is a rather
> rare incident. It's being used as part of the architecture of
> system nowadays.

Containers are not a problem. By definition they share the Kernel with
the host and therefore use the CPRNG of the host.

For early boot and VM Python needs to address the problem. Neither
Py_HashSecret nor the init vector of random's MT need strong
cryptographic keys. Python can ask the kernel if it has initialized its
RNG and then fall back to a different RNG source (e.g. srandom(3) /
random(3)).

For os.urandom() let's define it as non-blocking and raise an exception
when it would blocks. We can safely assume that the Kernel will never
block reads from its entropy source once the entropy source has been
initialized. Like Py_HashSecret we can ask the Kernel for its state (on
Linux: use non-blocking getrandom()).

The design is simple, secure and well specified. Python's startup works
under all circumstancesm os.urandom() either succeeds or fails properly
and 'import random' never blocks.

Christian
msg267998 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 09:10
> "early" is not correct: it's late in fact, only when the desktop is opened. I should use GPEDIT.MSC to start a task before the desktop, but this command doesn't seem to be available on my Windows 8?

Oh, I found how to start a task before the user login, and os.urandom() still works: it produces 16 bytes immediatly (in 0.0 second).

As FreeBSD, I'm not sure that Windows is really impacted by the issue #26839.

Latest article about Windows RNG: http://eprint.iacr.org/2007/419
"Cryptanalysis of the Random Number Generator of the Windows Operating System" (2007) by Leo Dorrendorf and Zvi Gutterman and Benny Pinkas

See also "Where does a Hyper-V guest get its entropy when generating a certificate authority key pair?":
http://security.stackexchange.com/questions/51538/where-does-a-hyper-v-guest-get-its-entropy-when-generating-a-certificate-authori
msg268000 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 09:15
Christian Heimes added the comment:
> I'm -1 on a block=True/False parameter. It makes the API more awkward and will make people move away from os.urandom() to a self-made RNG because they perceive os.urandom() as potential blocking.

Are you against the feature of the API (function name/parameter name)?
If it's just the API, could you please propose something better?

> Victor, you can use sysctl on all BSD and Linux to check if the RNG has been seeded. On Linux it is kernel.random.entropy_avail >= kernel.random.read_wakeup_threshold. On BSD it is kern.random.sys.seeded == 1.

I know for BSD, but I failed to find a way to get access to a FreeBSD
where kern.random.sys.seeded is still zero.

For Linux, is kernel.random.entropy_avail related to /dev/random? In
my tests, reading from /dev/urandom never blocks even before urandom
is initialized.

=> see msg267974
msg268001 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 09:15
> Oh, I found how to start a task before the user login, and os.urandom() still works: it produces 16 bytes immediatly (in 0.0 second).

Just to confirm: that's a fresh Windows VM, never been booted before ever?  If it had ever been booted before, it might be saving its entropy pools to the hard disk at shutdown.

If you do the experiment a second time with another copy of the same fresh VM, does it generate the same 16 bytes?
msg268002 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2016-06-09 09:17
> In my tests, reading from /dev/urandom never blocks even before urandom is initialized.

That's correct, and is the big difference between getrandom(0) and reading from /dev/urandom.  If "the entropy pool has not been initialized" (terminology from the man pages), getrandom(0) will block, but read(/dev/urandom) will return bytes from the urandom CPRNG before it's been initialized.  Which means they are some seriously low-quality not-very-random numbers.
msg268003 - (view) Author: Cory Benfield (Lukasa) * Date: 2016-06-09 09:18
> Those say that if you call getrandom(GRND_NONBLOCK) before the entropy
> pool has been initialized, it will return EAGAIN, but any time you read
> from /dev/urandom you will always get random data.


Yeah, so this is why the crypto folks were all up in arms about falling back to the /dev/urandom behaviour on Linux: Linux's /dev/urandom behaviour is really pretty dangerous.

In essence, on Linux, /dev/urandom will *always* return you some bytes, and their actual quality is entirely uncertain. This means that fundamentally without interrogating the kernel state before you read, you can't really be sure that /dev/urandom is safe for what you want it to be safe for.

But /dev/random isn't a suitable replacement in most cases, because /dev/random goes too far the other way: it has this hyper-paranoid notion about "entropy" that isn't really well-founded, and so sometimes it'll go on strike for a while.

This is why we've been pushing to keep hold of the os.urandom() implementation that CPython 3.5 now has: it irons out one of the most annoying warts in Linux RNGs. As Ted Tso has said elsewhere in this thread, /dev/urandom only exhibits its current behaviour for backward compatibility reasons: getrandom(flags=0) is really the ideal behaviour for almost any case *except* what Python is doing at startup.

Not that it needs saying again, but I'm still in favour of doing something like what Christian is suggesting, or like I suggested earlier (have os.urandom remain good, have Python internally fall back to weaker seeds for random and SipHash if it's run so early that the system hasn't started up fully yet).
msg268004 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 09:20
> Not that it needs saying again, but I'm still in favour of doing something like what Christian is suggesting, or like I suggested earlier (have os.urandom remain good, have Python internally fall back to weaker seeds for random and SipHash if it's run so early that the system hasn't started up fully yet).

It looks like people don't read what I'm writing :-( You are just
repeating the title of this issue :-) (So yes, I agree with you, since
I'm proposing the same thing.)
msg268006 - (view) Author: Cory Benfield (Lukasa) * Date: 2016-06-09 09:21
> It looks like people don't read what I'm writing :-( 

I'm reading you, Victor, but you and I aren't disagreeing, so I'm not trying to convince you. =) I'm trying to convince Larry.
msg268007 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 09:23
Christian Heimes:
> For os.urandom() let's define it as non-blocking and raise an exception
> when it would blocks.

There is a lot of discussion around random in the bug tracker, it's
really hard to follow :-( Please use specific issues for each idea.
Making os.urandom() non-blocking and always raise an exception is not
the goal of this issue. This idea explicitly makes os.urandom()
blocking on Linux. I suggest you to open a new issue.
msg268009 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-06-09 09:31
Larry Hastings:
> Just to confirm: that's a fresh Windows VM, never been booted before ever?  If it had ever been booted before, it might be saving its entropy pools to the hard disk at shutdown.

The VM was booted before. I don't see how I could schedule a task at boot, and then reboot... the new boot will obviously not be a "fresh VM".

Maybe it's possible to skip entropy written on disk on FreeBSD or Windows? If not, it confirms that the issue doesn't really affect FreeBSD and Windows in practice.

I read that OpenBSD is able to pass the entropy file through the boot loader. It is done before the kernel is loaded, so it doesn't matter when Python 3.5 is started, urandom will always be initialized after the first boot on OpenBSD, no? (If the first boot was able to produce enough entropy.) Maybe it's the same thing for FreeBSD.

Linux has a different design, loading the entropy file from the disk comes "later" in the init process, after the kernel booted. It's not done (currently) by the boot loader. It was discussed at:
http://bugs.python.org/issue26839#msg267853


> If you do the experiment a second time with another copy of the same fresh VM, does it generate the same 16 bytes?

From what I read, Windows is vulnerable the "reset" attack on the RNG when using a VM. So you can expect the same random numbers with your scenario.
msg268017 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-06-09 11:18
Having os.urandom raise an error instead of blocking is OK with me. It turns an implicit error into an explicit one. However, I prefer to have it block until it has initialized it's entropy pool because that makes Linux behave similarly to all of the other major, modern OSs instead of making Linux continue to be a weird edge case.
msg268043 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-06-09 17:21
Since Victor requested it, I filed #27282 to track the "raise BlockingIOError if the kernel would block" design option.

The key advantage that particular model offers is that it's trivial to build a blocking version as a busy loop around the non-blocking version:

    def urandom_wait_for_entropy(num_bytes):
        while True:
            try:
                return os.urandom(num_bytes)
            except BlockingIOError:
                pass

And if you ignore the problem and just call os.urandom(), you'll almost certainly be fine unless you're working with Linux boot scripts or embedded ARM devices (in which case, this point will be minor compared to the other arcana you're dealing with).
msg271570 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-07-28 17:07
I don't like this idea anymore ("add block=False parameter to os.urandom()"), so I close this issue.

I wrote the PEP 524 to propose to make os.urandom() blocking on Linux. The discussion moved to the security-sig mailing list.
History
Date User Action Args
2022-04-11 14:58:32adminsetgithub: 71453
2016-07-28 17:07:55vstinnersetstatus: open -> closed
resolution: rejected
messages: + msg271570
2016-06-16 13:26:47dstufftsetnosy: - dstufft
2016-06-12 11:34:28christian.heimessetnosy: - christian.heimes
2016-06-09 17:21:54ncoghlansetmessages: + msg268043
2016-06-09 11:18:48dstufftsetmessages: + msg268017
2016-06-09 09:31:07vstinnersetmessages: + msg268009
2016-06-09 09:23:54vstinnersetmessages: + msg268007
2016-06-09 09:21:46Lukasasetmessages: + msg268006
2016-06-09 09:20:57vstinnersetmessages: + msg268004
2016-06-09 09:18:08Lukasasetmessages: + msg268003
2016-06-09 09:17:28larrysetmessages: + msg268002
2016-06-09 09:15:40larrysetmessages: + msg268001
2016-06-09 09:15:08vstinnersetmessages: + msg268000
2016-06-09 09:10:19vstinnersetmessages: + msg267998
2016-06-09 09:06:51christian.heimessetmessages: + msg267997
2016-06-09 08:49:15lemburgsetmessages: + msg267996
2016-06-09 08:41:16vstinnersetmessages: + msg267994
2016-06-09 08:41:11larrysetmessages: + msg267993
2016-06-09 08:19:06vstinnersetmessages: + msg267988
2016-06-09 08:16:42vstinnersetmessages: + msg267987
2016-06-09 08:14:40Lukasasetmessages: + msg267985
2016-06-09 08:11:37lemburgsetmessages: + msg267982
2016-06-09 08:11:14christian.heimessetmessages: + msg267981
2016-06-09 08:10:27larrysetmessages: + msg267980
2016-06-09 08:08:52vstinnersetmessages: + msg267979
2016-06-09 08:04:16Lukasasetnosy: + Lukasa
2016-06-09 08:02:15vstinnersetmessages: + msg267976
2016-06-09 07:57:55vstinnersetmessages: + msg267974
2016-06-09 07:57:49larrysetmessages: + msg267973
2016-06-09 07:56:01lemburgsetnosy: + lemburg
messages: + msg267971
2016-06-09 07:41:22christian.heimessetnosy: + christian.heimes
messages: + msg267968
2016-06-09 07:31:26martin.pantersetnosy: + martin.panter
messages: + msg267966
2016-06-09 05:51:03larrysetmessages: + msg267963
2016-06-09 05:48:09vstinnersetmessages: + msg267962
2016-06-09 01:50:16larrysetmessages: + msg267948
2016-06-09 01:08:21vstinnersetmessages: + msg267947
2016-06-09 00:35:18dstufftsetmessages: + msg267945
2016-06-09 00:34:01larrysetmessages: + msg267944
2016-06-09 00:29:34larrysetmessages: + msg267943
2016-06-09 00:24:46larrysetmessages: + msg267942
2016-06-09 00:15:05ncoghlansetmessages: + msg267941
2016-06-09 00:14:27dstufftsetmessages: + msg267940
2016-06-09 00:04:54ncoghlansetmessages: + msg267938
2016-06-09 00:03:11larrysetmessages: + msg267937
2016-06-08 23:59:22larrysetmessages: + msg267935
2016-06-08 23:51:58dstufftsetmessages: + msg267934
2016-06-08 23:43:53larrysetmessages: + msg267931
2016-06-08 23:32:33larrysetmessages: + msg267930
2016-06-08 23:31:38ncoghlansetnosy: + ncoghlan
2016-06-08 23:16:00dstufftsetmessages: + msg267927
2016-06-08 23:14:41dstufftsetmessages: + msg267926
2016-06-08 23:14:17larrysetmessages: + msg267925
2016-06-08 22:58:18dstufftsetmessages: + msg267924
2016-06-08 22:56:24dstufftsetmessages: + msg267923
2016-06-08 22:54:20larrysetmessages: + msg267922
2016-06-08 22:33:02vstinnerlinkissue27249 superseder
2016-06-08 22:29:41vstinnersetmessages: + msg267917
2016-06-08 20:00:44larrysetmessages: + msg267883
2016-06-08 19:54:46vstinnersetmessages: + msg267882
2016-06-08 19:02:15Theodore Tsosetnosy: + Theodore Tso
messages: + msg267879
2016-06-08 18:40:30larrysetmessages: + msg267878
2016-06-08 16:24:19vstinnersettitle: Always use getrandom() in os.random() and add block=False parameter to os.urandom() -> Always use getrandom() in os.random() on Linux and add block=False parameter to os.urandom()
2016-06-08 16:23:19vstinnersettype: enhancement
title: Add block keyword-only optional parameter to os.urandom() -> Always use getrandom() in os.random() and add block=False parameter to os.urandom()
messages: + msg267868
versions: - Python 3.5
2016-06-08 16:18:34vstinnersetmessages: + msg267867
2016-06-08 10:06:01dstufftsetnosy: + dstufft
messages: + msg267829
2016-06-08 10:03:36vstinnersetmessages: + msg267828
2016-06-08 09:45:16vstinnersetmessages: + msg267824
2016-06-08 09:43:07vstinnersetnosy: + larry
2016-06-08 09:43:02vstinnersetmessages: + msg267821
2016-06-08 09:19:03vstinnercreate