classification
Title: Provide a way to enable getrandom on Linux even when build system runs an older kernel
Type: enhancement Stage: needs patch
Components: Versions: Python 3.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, ncoghlan, petr.viktorin, vstinner
Priority: normal Keywords:

Created on 2016-09-07 02:46 by ncoghlan, last changed 2016-09-21 12:55 by vstinner. This issue is now closed.

Messages (9)
msg274715 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-09-07 02:46
The configure script determines the setting for HAVE_GETRANDOM_SYSCALL at build time, which means the dynamic check for getrandom() support in the Linux kernel gets disabled when building against an older kernel.

This impacts the implicit use of getrandom() in os.urandom(): https://mail.python.org/pipermail/security-sig/2016-June/000060.html

And also the new os.getrandom() API added in #27778: http://bugs.python.org/issue27778#msg274698

It's desirable to have a way of forcing the inclusion of the dynamic runtime check, even if the currently running kernel doesn't provide the syscall itself.
msg274727 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-07 03:07
Support for the getrandom() syscall requires syscall(), SYS_getrandom constant, GRND_NONBLOCK and GRND_RANDOM constants, linux/random.h header, etc.

I don't understand your request. configure doesn't check getrandom() syscall result:

        /* ignore the result, Python checks for ENOSYS and EAGAIN at runtime */
        (void)syscall(SYS_getrandom, buffer, buflen, flags);

I don't see how you can support getrandom() if the host doesn't have all constants to compile the C code. Do you have to hardcode constants?
msg274758 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-09-07 04:39
It may be that the right fix is to build with more recent Linux header files, even when running the build on an older kernel, in which case the request would be to have a way to indicate that missing those particular headers should fail the build, rather than implicitly omitting os.getrandom().

Worst case, yeah, we'll have to patch Fedora's system Python to hardcode the necessary constants in order to cope with the way Koji's buildroots are currently set up. I hope it doesn't come to that, though.

I'll leave the specifics to Petr, but it would definitely be preferable to have a common way of solving this recommended by upstream rather than everyone affected needing to figure out how to solve the problem on their own.
msg277072 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-20 21:14
I'm not excited by the idea of using hardcoded constants for getrandom(). There is a risk of using wrong constants depending on the architecture or the Linux kernel version.

The code is already very low-level: it calls directly the syscall using syscall() function. getrandom() has no nice glibc clean API yet:
https://sourceware.org/bugzilla/show_bug.cgi?id=17252

I suggest to close the issue as WONTFIX. It's ok to use a file descriptor and read /dev/urandom. getrandom() is nice to have, but it's not really a killer feature.

For Fedora: sure, you can use vendor patches to workaround your technical issue, builders using an old kernel :-) But it would be simpler to upgrade the builder, no? :-)
msg277113 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-09-21 09:02
I'm happy to close this as not even being a bug - the Fedora Python maintainers found that it was only specifically 3.5.1 that wasn't finding the necessary header files in the Fedora build root (and hence not even trying the syscall), while 3.5.0 and 3.5.2 were both fine.
msg277120 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-21 10:19
Oh it's strange that only 3.5.1 has the issue. Happy to read that the
problem is already solved!
msg277122 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2016-09-21 11:05
Indeed, I'd close WONTFIX.

IMO, applications that:
- run at early boot, and
- get built with an older kernel than they run on
fall squarely into the enterprise distro turf, and CPython code shouldn't include hacks needed to make this work. That's not to say I'm opposed to collaboration if anyone else needs to solve this problem – quite the opposite. But I think a vendor-specific patch is the right layer here.

As currently documented, the getrandom call is not used on older kernels.

(Btw, no, it's not easy for *me* to help upgrading Fedora builders. But I can easily maintain a vendor patch for Python, if it comes to that.)
msg277123 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2016-09-21 11:09
(Please ignore that comment – I was on vacation and, when clearing my backlog, got to this issue before the Fedora discussions.)
msg277131 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-21 12:55
Ok, I changed the status to WONTFIX.
History
Date User Action Args
2016-09-21 12:55:09vstinnersetresolution: not a bug -> wont fix
messages: + msg277131
2016-09-21 11:09:34petr.viktorinsetmessages: + msg277123
2016-09-21 11:05:33petr.viktorinsetmessages: + msg277122
2016-09-21 10:19:05vstinnersetmessages: + msg277120
2016-09-21 09:02:30ncoghlansetstatus: open -> closed
resolution: not a bug
messages: + msg277113
2016-09-20 21:14:17vstinnersetmessages: + msg277072
2016-09-07 04:39:11ncoghlansetmessages: + msg274758
2016-09-07 03:07:50vstinnersetnosy: + vstinner
messages: + msg274727
2016-09-07 02:46:39ncoghlancreate