classification
Title: os.urandom(2500) fails on Solaris 11.3
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mryan1539, python-dev, rhettinger, vstinner
Priority: normal Keywords: patch

Created on 2016-04-11 17:35 by mryan1539, last changed 2016-04-12 20:40 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
python3-getrandom.patch mryan1539, 2016-04-11 17:35
urandom_solaris.patch vstinner, 2016-04-12 08:15 review
Messages (8)
msg263191 - (view) Author: Matthew Ryan (mryan1539) * Date: 2016-04-11 17:35
On Solaris 11.3 (intel tested, but I assume issue is on SPARC as well),
I found the following fails:
  import os
  os.urandom(2500)

The above throws OSError: [Errno 22] Invalid argument.

It turns out that the Solaris version of getrandom() is limited to returning
no more than 1024 bytes, per the manpage:
  The getrandom() and getentropy() functions fail if:

  EINVAL    The flags are not set to GRND_RANDOM, GRND_NONBLOCK or  both,
            or bufsz is <= 0 or > 1024.

I've attached a possible patch for this issue, against the 3.5.1 source
tree.
msg263235 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-04-12 08:03
See also the issue #25003 and the changeset 835085cc28cd:

Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the getrandom() function instead of the getentropy() function. The getentropy() function is blocking to generate very good quality entropy, os.urandom() doesn't need such high-quality entropy.
msg263236 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-04-12 08:15
> I've attached a possible patch for this issue, against the 3.5.1 source
tree.

I guess that you are already using Python 3.5.1 which uses getrandom(). You should try to confirm using strace.

I updated your patch. I replaced "#if defined(__sun__)" with "#ifdef sun", since "#ifdef sun" looks more common in the Python code base, and I never saw "#if defined(__sun__)" in the Python code base.

I also avoided the new len variable, I reused the n variable.

I don't have Solaris, so I cannot test. I didn't find getrandom() manual page neither, I only found this blog post which doesn't mention the 1024 bytes limitation on Solaris:
https://blogs.oracle.com/darren/entry/solaris_new_system_calls_getentropy
msg263259 - (view) Author: Matthew Ryan (mryan1539) * Date: 2016-04-12 15:57
The new patch looks fine; I used __sun__ rather than sun out of habit
(C standard requires
system specific macros be in the reserved namespace), but either will work.

I found the original problem through debugging with GDB, so I know
getrandom() was being
called, and the test case I provided (taken from Lib/Random.py) fails
without the patch,
and succeeds with it.

Oddly, the blog post you linked to describes getrandom as:
"Recent Linux kernels have a getrandom(2) system call that reads
between 1 and 1024 bytes of randomness"

But no such limit current exists in the Linux version that I can see;
however, the Solaris
version definitely does have that limit:
https://docs.oracle.com/cd/E53394_01/html/E54765/getrandom-2.html

On Tue, Apr 12, 2016 at 1:15 AM, STINNER Victor <report@bugs.python.org> wrote:
>
> STINNER Victor added the comment:
>
>> I've attached a possible patch for this issue, against the 3.5.1 source
> tree.
>
> I guess that you are already using Python 3.5.1 which uses getrandom(). You should try to confirm using strace.
>
> I updated your patch. I replaced "#if defined(__sun__)" with "#ifdef sun", since "#ifdef sun" looks more common in the Python code base, and I never saw "#if defined(__sun__)" in the Python code base.
>
> I also avoided the new len variable, I reused the n variable.
>
> I don't have Solaris, so I cannot test. I didn't find getrandom() manual page neither, I only found this blog post which doesn't mention the 1024 bytes limitation on Solaris:
> https://blogs.oracle.com/darren/entry/solaris_new_system_calls_getentropy
>
> ----------
> Added file: http://bugs.python.org/file42443/urandom_solaris.patch
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26735>
> _______________________________________
msg263261 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-04-12 16:12
> The new patch looks fine

Do you mean that it fixes your issue? Can it be applied to Python 3.5 & 3.6?
msg263269 - (view) Author: Matthew Ryan (mryan1539) * Date: 2016-04-12 18:29
Yes, I've verified that:
* the issue existed in the default branch as of this morning.
* the patch applies cleanly against both 3.5 and default, and
addresses the issue in both branches.
msg263274 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-04-12 20:39
New changeset fb7628e8dfef by Victor Stinner in branch '3.5':
Fix os.urandom() on Solaris 11.3
https://hg.python.org/cpython/rev/fb7628e8dfef
msg263275 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-04-12 20:40
> Yes, I've verified that: (...)

Cool, thanks for the bug report and for the check.

It's now fixed. In the meanwhile, you can workaround the issue by limiting yourself calls to os.urandom() to 1024 bytes (and then concatenate the result).
History
Date User Action Args
2016-04-12 20:40:59vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg263275
2016-04-12 20:39:57python-devsetnosy: + python-dev
messages: + msg263274
2016-04-12 18:29:30mryan1539setmessages: + msg263269
2016-04-12 16:12:59vstinnersetmessages: + msg263261
2016-04-12 15:57:52mryan1539setmessages: + msg263259
2016-04-12 08:15:47vstinnersetfiles: + urandom_solaris.patch

messages: + msg263236
2016-04-12 08:03:56vstinnersetmessages: + msg263235
2016-04-12 06:23:54serhiy.storchakasetnosy: + rhettinger
2016-04-12 06:17:32SilentGhostsetnosy: + vstinner

components: + Interpreter Core
stage: patch review
2016-04-11 17:35:08mryan1539create