Now that the ssl module exposes RAND_bytes() (issue #12049), it might be interesting to implement a SSLRandom generator to the random module, to have a cryptographically strong generator without the os.urandom() overhead.

Thoughts?

I'm -1

It doesn't look like OpenSSL's PRNG is much faster than /dev/urandom if Python keeps a persistent fd:

$ python3.3 prng.py 
loops: 1000000
bytes: 16
/dev/urandom (fd)        1.2181
os.urandom()             3.2892
RAND_pseudo_bytes        1.2924
RAND_bytes               1.2945

Some crypto experts also say that the PRNG of a stock OpenSSL build is sub par. The algorithm is neither standardized by NIST nor carefully developed by a cryptographer. FIPS builds of OpenSSL have a better PRNG.

> It doesn't look like OpenSSL's PRNG is much faster than /dev/urandom if
> Python keeps a persistent fd:
>
> $ python3.3 prng.py
> loops: 1000000
> bytes: 16
> /dev/urandom (fd)        1.2181
> os.urandom()             3.2892
> RAND_pseudo_bytes        1.2924
> RAND_bytes               1.2945

Except that your benchmark uses 4096-bytes buffering for /dev/urandom
with a persistent FD, which is not the case with the implementation
proposed in #issue18756. Without buffering (i.e. one read() syscall
per invokation), the numbers change:

$ ./python  ~/prng.py
loops: 100000
bytes: 16
/dev/urandom (fd)        1.37106
os.urandom()             1.4446
RAND_pseudo_bytes        0.83635
RAND_bytes               0.75604

Also, you use CLOCK_PROCESS_CPUTIME_ID which only accounts for CPU
time, not wall-clock time, but it doesn't change much.

> Some crypto experts also say that the PRNG of a stock OpenSSL build is sub
> par. The algorithm is neither standardized by NIST nor carefully developed
> by a cryptographer. FIPS builds of OpenSSL have a better PRNG.

That's an implementation detail, it could very well improve.

But I don't have a strong feeling, so if that's not deemed useful, I
won't take it personally :-)

Use time.perf_counter() for benchmarkks!!!

It would be interesting to compare performances on other platforms:
Windows, Solaris, Mac OS X, ...

Improved benchmark script:

Loops: 500000           1       4       8       16      32      64      128
urandom (fd)            0.091   0.185   0.306   0.544   1.060   1.989   3.876
urandom (fd, no buf)    0.504   0.502   0.502   0.806   1.398   2.324   4.086
os.urandom()            1.155   1.154   1.154   1.498   2.052   2.975   4.910
RAND_pseudo_bytes       0.436   0.436   0.435   0.555   0.753   1.064   1.664
RAND_bytes              0.412   0.417   0.417   0.534   0.741   1.046   1.708

Seriously, why are we obsessed with performance when talking about a security feature? Did anyone *actually* complain about urandom() being too slow (ok, someone complained about it eating file descriptors... :-))?

The question is whether OpenSSL's random generator produces better randomness than the system one: apparently it's not the case under Linux, how about other systems?

I don't get it either. urandom is perfectly fine. I showed that urandom is just a bit slower than SSL_rand().

How about we generalize SystemRandom so  users can implement a custom RNG class wby providing a method rng(amount) -> bytes?

Antoine Pitrou <report@bugs.python.org> schrieb:
>
>Antoine Pitrou added the comment:
>
>Seriously, why are we obsessed with performance when talking about a
>security feature? Did anyone *actually* complain about urandom() being
>too slow (ok, someone complained about it eating file descriptors...
>:-))?
>
>The question is whether OpenSSL's random generator produces better
>randomness than the system one: apparently it's not the case under
>Linux, how about other systems?
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue18811>
>_______________________________________

> How about we generalize SystemRandom so  users can implement
> a custom RNG class wby providing a method rng(amount) -> bytes?

The random module already makes it easy:
"""
Class Random can also be subclassed if you want to use a different basic generator of your own devising: in that case, override the random(), seed(), getstate(), and setstate() methods. Optionally, a new generator can supply a getrandbits() method — this allows randrange() to produce selections over an arbitrarily large range.
"""

> I showed that urandom is just a bit slower than SSL_rand().

"Just a bit" is relative, I read from 2 to 3 times slower, on Linux.

> I don't get it either. urandom is perfectly fine.

Alright, closing then.

> How about we generalize SystemRandom so  users can implement a custom
> RNG class wby providing a method rng(amount) -> bytes?

We may make it easier to implement custom Random implementations with
user-defined random generators, but they needn't be *SystemRandom*
subclasses ;-)