Message379568
Performance improvement measured on a 1.4Ghz quad aarch64 A57 (nvidia jetson nano):
#define VFORK_USABLE 1
test_subprocess: 36.5 seconds
#undef VFORK_USABLE
test_subprocess: 45 seconds
Nice. I really didn't expect a 20% speedup on its testsuite alone!
Lets dive into that with a microbenchmark:
$ ./build-novfork/python ten_seconds_of_truth.py
Launching /bin/true for 10 seconds in a loop.
Launched 2713 subprocesses in 10.00194378796732 seconds.
271.247275281014 subprocesses/second.
Increased our mapped pages by 778240KiB.
Launching /bin/true for 10 seconds in a loop.
Launched 212 subprocesses in 10.006392606999725 seconds.
21.186456331095847 subprocesses/second.
$ ./build/python ten_seconds_of_truth.py
Launching /bin/true for 10 seconds in a loop.
Launched 3310 subprocesses in 10.001623224001378 seconds.
330.94628000551285 subprocesses/second.
Increased our mapped pages by 778240KiB.
Launching /bin/true for 10 seconds in a loop.
Launched 3312 subprocesses in 10.001519071985967 seconds.
331.1496959773679 subprocesses/second.
Demonstrating perfectly the benefit of vfork(). The more mapped pages, the slower regular fork() becomes. With vfork() the size of the process's memory map does not matter.
ten_seconds_of_truth.py:
```python
from time import monotonic as now
import subprocess
def benchmark_for_ten():
print('Launching /bin/true for 10 seconds in a loop.')
count = 0
start = now()
while now() - start < 10.:
subprocess.run(['/bin/true'])
count += 1
end = now()
duration = end-start
print(f'Launched {count} subprocesses in {duration} seconds.')
print(f'{count/duration} subprocesses/second.')
benchmark_for_ten()
map_a_bunch_of_pages = '4agkahglahaa^#\0ag3\3'*1024*1024*40
print(f'Increased our mapped pages by {len(map_a_bunch_of_pages)//1024}KiB.')
benchmark_for_ten()
``` |
|
Date |
User |
Action |
Args |
2020-10-25 06:38:16 | gregory.p.smith | set | recipients:
+ gregory.p.smith, ronaldoussoren, koobs, izbyshev, pablogsal, Yonatan Goldschmidt |
2020-10-25 06:38:16 | gregory.p.smith | set | messageid: <1603607896.72.0.37134453627.issue35823@roundup.psfhosted.org> |
2020-10-25 06:38:16 | gregory.p.smith | link | issue35823 messages |
2020-10-25 06:38:16 | gregory.p.smith | create | |
|