Author vstinner
Recipients gregory.p.smith, izbyshev, koobs, nanjekyejoannah, pablogsal, serhiy.storchaka, vstinner
Date 2019-01-15.23:28:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1547594910.44.0.165167247722.issue35537@roundup.psfhosted.org>
In-reply-to
Content
More benchmarks.

I modified subprocess_bench.py to use:
ARGS = ["/usr/bin/python3", "-S", "-E", "-c", "pass"]
=> Mean +- std dev: [fork_exec] 34.1 ms +- 0.4 ms -> [posix_spawn] 6.85 ms +- 0.08 ms: 4.97x faster (-80%)

Benchmark using:
ARGS = ["/usr/bin/python3", "-c", "pass"]
=> Mean +- std dev: [fork_exec] 44.5 ms +- 0.6 ms -> [posix_spawn] 17.2 ms +- 0.2 ms: 2.58x faster (-61%)

Copy of the previous benchmark using /usr/bin/true:
Mean +- std dev: [fork_exec] 27.1 ms +- 0.4 ms -> [posix_spawn] 447 us +- 163 us: 60.55x faster (-98%)

The benchmark is less impressive with Python which has a longer startup time (7 to 17 ms depending on the -S option).

The speedup is between 2.6x (default) and 5.0x (-S option) faster for Python... to execute "pass" (do nothing).

In short, I understand that vfork removes a fixed cost of 27 ms which is the cost of duplicating the 2 GiB of memory pages.

The speedup depends on the memory footprint of the parent process and the execution time of the child process. The best speedup is when the parent is the largest and the child is the fastest.

--

Another benchmark, I modified subprocess_bench.py with:

BIG_ALLOC = b'x' * (10 * 1024 * 1024 * 1024)   # 10 GiB
ARGS = ["/bin/true"]

Mean +- std dev: [fork_exec] 139 ms +- 9 ms -> [posix_spawn] 420 us +- 208 us: 331.40x faster (-100%)

Here the cost of copying 10 GiB of memory pages is around 138 ms. It's 331x faster ;-)
History
Date User Action Args
2019-01-15 23:28:32vstinnersetrecipients: + vstinner, gregory.p.smith, serhiy.storchaka, koobs, izbyshev, pablogsal, nanjekyejoannah
2019-01-15 23:28:30vstinnersetmessageid: <1547594910.44.0.165167247722.issue35537@roundup.psfhosted.org>
2019-01-15 23:28:30vstinnerlinkissue35537 messages
2019-01-15 23:28:30vstinnercreate