This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients mark.dickinson, rhettinger, serhiy.storchaka, veky, vstinner
Date 2020-04-17.17:03:16
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1587142997.1.0.61164420414.issue40286@roundup.psfhosted.org>
In-reply-to
Content
I wrote a quick benchmark:
---
import pyperf
import random

gen = random.Random()
# gen = random.SystemRandom()
gen.seed(850779834)

if 1: #hasattr(gen, 'randbytes'):
    func = type(gen).randbytes
elif 0:
    def py_randbytes(gen, n):
        data = bytearray(n)
        i = 0
        while i < n:
            chunk = 4
            word = gen.getrandbits(32)
            word = word.to_bytes(4, 'big')
            chunk = min(n, 4)
            data[i:i+chunk] = word[:chunk]
            i += chunk
        return bytes(data)

    func = py_randbytes
else:
    def getrandbits_to_bytes(gen, n):
        return gen.getrandbits(n * 8).to_bytes(n, 'little')

    func = getrandbits_to_bytes

runner = pyperf.Runner()
for nbytes in (1, 4, 16, 1024, 1024 * 1024):
    runner.bench_func(f'randbytes({nbytes})', func, gen, nbytes)
---

Results on Linux using gcc -O3 (without LTO or PGO) using the C randbytes() implementation as the reference:

+--------------------+-------------+----------------------------------+-------------------------------+
| Benchmark          | c_randbytes | py_randbytes                     | getrandbits_to_bytes          |
+====================+=============+==================================+===============================+
| randbytes(1)       | 71.4 ns     | 1.04 us: 14.51x slower (+1351%)  | 244 ns: 3.42x slower (+242%)  |
+--------------------+-------------+----------------------------------+-------------------------------+
| randbytes(4)       | 71.4 ns     | 1.03 us: 14.48x slower (+1348%)  | 261 ns: 3.66x slower (+266%)  |
+--------------------+-------------+----------------------------------+-------------------------------+
| randbytes(16)      | 81.9 ns     | 3.07 us: 37.51x slower (+3651%)  | 321 ns: 3.92x slower (+292%)  |
+--------------------+-------------+----------------------------------+-------------------------------+
| randbytes(1024)    | 1.05 us     | 173 us: 165.41x slower (+16441%) | 3.66 us: 3.49x slower (+249%) |
+--------------------+-------------+----------------------------------+-------------------------------+
| randbytes(1048576) | 955 us      | 187 ms: 196.30x slower (+19530%) | 4.37 ms: 4.58x slower (+358%) |
+--------------------+-------------+----------------------------------+-------------------------------+

* c_randbytes: PR 19527, randbytes() methods implemented in C
* py_randbytes: bytearray, getrandbits(), .to_bytes()
* getrandbits_to_bytes: Serhiy's implementation: gen.getrandbits(n * 8).to_bytes(n, 'little')

So well, the C randbytes() implementation is always the fastest.


random.SystemRandom().randbytes() (os.urandom(n)) performance using random.Random().randbytes() (Mersenne Twister) as a reference:

+--------------------+-------------+---------------------------------+
| Benchmark          | c_randbytes | systemrandom                    |
+====================+=============+=================================+
| randbytes(1)       | 71.4 ns     | 994 ns: 13.93x slower (+1293%)  |
+--------------------+-------------+---------------------------------+
| randbytes(4)       | 71.4 ns     | 1.04 us: 14.60x slower (+1360%) |
+--------------------+-------------+---------------------------------+
| randbytes(16)      | 81.9 ns     | 1.02 us: 12.49x slower (+1149%) |
+--------------------+-------------+---------------------------------+
| randbytes(1024)    | 1.05 us     | 6.22 us: 5.93x slower (+493%)   |
+--------------------+-------------+---------------------------------+
| randbytes(1048576) | 955 us      | 5.64 ms: 5.91x slower (+491%)   |
+--------------------+-------------+---------------------------------+

os.urandom() is way slower than Mersenne Twister.

Well, that's not surprising: os.urandom() requires at least one syscall (getrandom() syscall on my Linux machine).
History
Date User Action Args
2020-04-17 17:03:17vstinnersetrecipients: + vstinner, rhettinger, mark.dickinson, serhiy.storchaka, veky
2020-04-17 17:03:17vstinnersetmessageid: <1587142997.1.0.61164420414.issue40286@roundup.psfhosted.org>
2020-04-17 17:03:17vstinnerlinkissue40286 messages
2020-04-17 17:03:16vstinnercreate