This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: add ssl-based generator to random module
Type: enhancement Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: christian.heimes, neologix, pitrou, rhettinger, vstinner
Priority: normal Keywords:

Created on 2013-08-22 17:04 by neologix, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
prng.py christian.heimes, 2013-08-22 23:40
Messages (9)
msg195912 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-22 17:04
Now that the ssl module exposes RAND_bytes() (issue #12049), it might be interesting to implement a SSLRandom generator to the random module, to have a cryptographically strong generator without the os.urandom() overhead.

Thoughts?
msg195913 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-08-22 17:45
I'm -1

It doesn't look like OpenSSL's PRNG is much faster than /dev/urandom if Python keeps a persistent fd:

$ python3.3 prng.py 
loops: 1000000
bytes: 16
/dev/urandom (fd)        1.2181
os.urandom()             3.2892
RAND_pseudo_bytes        1.2924
RAND_bytes               1.2945

Some crypto experts also say that the PRNG of a stock OpenSSL build is sub par. The algorithm is neither standardized by NIST nor carefully developed by a cryptographer. FIPS builds of OpenSSL have a better PRNG.
msg195914 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-22 18:47
> It doesn't look like OpenSSL's PRNG is much faster than /dev/urandom if
> Python keeps a persistent fd:
>
> $ python3.3 prng.py
> loops: 1000000
> bytes: 16
> /dev/urandom (fd)        1.2181
> os.urandom()             3.2892
> RAND_pseudo_bytes        1.2924
> RAND_bytes               1.2945

Except that your benchmark uses 4096-bytes buffering for /dev/urandom
with a persistent FD, which is not the case with the implementation
proposed in #issue18756. Without buffering (i.e. one read() syscall
per invokation), the numbers change:

$ ./python  ~/prng.py
loops: 100000
bytes: 16
/dev/urandom (fd)        1.37106
os.urandom()             1.4446
RAND_pseudo_bytes        0.83635
RAND_bytes               0.75604

Also, you use CLOCK_PROCESS_CPUTIME_ID which only accounts for CPU
time, not wall-clock time, but it doesn't change much.

> Some crypto experts also say that the PRNG of a stock OpenSSL build is sub
> par. The algorithm is neither standardized by NIST nor carefully developed
> by a cryptographer. FIPS builds of OpenSSL have a better PRNG.

That's an implementation detail, it could very well improve.

But I don't have a strong feeling, so if that's not deemed useful, I
won't take it personally :-)
msg195921 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-08-22 21:18
Use time.perf_counter() for benchmarkks!!!

It would be interesting to compare performances on other platforms:
Windows, Solaris, Mac OS X, ...
msg195933 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-08-22 23:40
Improved benchmark script:

Loops: 500000           1       4       8       16      32      64      128
urandom (fd)            0.091   0.185   0.306   0.544   1.060   1.989   3.876
urandom (fd, no buf)    0.504   0.502   0.502   0.806   1.398   2.324   4.086
os.urandom()            1.155   1.154   1.154   1.498   2.052   2.975   4.910
RAND_pseudo_bytes       0.436   0.436   0.435   0.555   0.753   1.064   1.664
RAND_bytes              0.412   0.417   0.417   0.534   0.741   1.046   1.708
msg195956 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-23 09:15
Seriously, why are we obsessed with performance when talking about a security feature? Did anyone *actually* complain about urandom() being too slow (ok, someone complained about it eating file descriptors... :-))?

The question is whether OpenSSL's random generator produces better randomness than the system one: apparently it's not the case under Linux, how about other systems?
msg195974 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-08-23 14:28
I don't get it either. urandom is perfectly fine. I showed that urandom is just a bit slower than SSL_rand().

How about we generalize SystemRandom so  users can implement a custom RNG class wby providing a method rng(amount) -> bytes?

Antoine Pitrou <report@bugs.python.org> schrieb:
>
>Antoine Pitrou added the comment:
>
>Seriously, why are we obsessed with performance when talking about a
>security feature? Did anyone *actually* complain about urandom() being
>too slow (ok, someone complained about it eating file descriptors...
>:-))?
>
>The question is whether OpenSSL's random generator produces better
>randomness than the system one: apparently it's not the case under
>Linux, how about other systems?
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue18811>
>_______________________________________
msg195975 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-23 14:34
> How about we generalize SystemRandom so  users can implement
> a custom RNG class wby providing a method rng(amount) -> bytes?

The random module already makes it easy:
"""
Class Random can also be subclassed if you want to use a different basic generator of your own devising: in that case, override the random(), seed(), getstate(), and setstate() methods. Optionally, a new generator can supply a getrandbits() method — this allows randrange() to produce selections over an arbitrarily large range.
"""

> I showed that urandom is just a bit slower than SSL_rand().

"Just a bit" is relative, I read from 2 to 3 times slower, on Linux.

> I don't get it either. urandom is perfectly fine.

Alright, closing then.
msg195976 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-23 14:36
> How about we generalize SystemRandom so  users can implement a custom
> RNG class wby providing a method rng(amount) -> bytes?

We may make it easier to implement custom Random implementations with
user-defined random generators, but they needn't be *SystemRandom*
subclasses ;-)
History
Date User Action Args
2022-04-11 14:57:49adminsetgithub: 63011
2013-08-23 14:36:16pitrousetmessages: + msg195976
2013-08-23 14:34:55neologixsetstatus: open -> closed
resolution: rejected
messages: + msg195975

stage: needs patch -> resolved
2013-08-23 14:28:20christian.heimessetmessages: + msg195974
2013-08-23 09:15:05pitrousetmessages: + msg195956
2013-08-23 01:46:29rhettingersetassignee: rhettinger

nosy: + rhettinger
2013-08-23 00:07:27christian.heimessetfiles: - prng.py
2013-08-22 23:40:46christian.heimessetfiles: + prng.py

messages: + msg195933
2013-08-22 21:18:20vstinnersetmessages: + msg195921
2013-08-22 18:47:26neologixsetmessages: + msg195914
2013-08-22 17:45:24christian.heimessetfiles: + prng.py

messages: + msg195913
versions: + Python 3.4
2013-08-22 17:04:00neologixcreate