Author vstinner
Recipients Dennis Sweeney, Zeturic, ammar2, corona10, josh.r, pmpp, serhiy.storchaka, tim.peters, vstinner
Date 2020-10-13.21:04:17
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1602623057.51.0.41332997237.issue41972@roundup.psfhosted.org>
In-reply-to
Content
I compared PR 22679 using the commit 77f0a23e7a9fb247101b9b14a060c4ba1c4b87a5 as the reference using random_bench.py.

For a bunch of cases it's slower, for some others it's faster.

My modified pyperf computes a geometric mean of: 0.70 (faster).

$ PYTHONPATH=~/myprojects/pyperf/ python3 -m pyperf compare_to ref.json pr22679.json -G
Slower (4):
- needle=8: 816 us +- 227 us -> 1.13 ms +- 0.60 ms: 1.38x slower (+38%)
- needle=16: 579 us +- 208 us -> 780 us +- 391 us: 1.35x slower (+35%)
- needle=7: 865 us +- 301 us -> 1.15 ms +- 0.70 ms: 1.33x slower (+33%)
- needle=9: 827 us +- 250 us -> 1.04 ms +- 0.53 ms: 1.26x slower (+26%)

Faster (18):
- needle=3442: 2.24 ms +- 1.17 ms -> 846 us +- 895 us: 2.65x faster (-62%)
- needle=301: 1.72 ms +- 1.13 ms -> 652 us +- 774 us: 2.64x faster (-62%)
- needle=5164: 2.51 ms +- 1.24 ms -> 991 us +- 972 us: 2.53x faster (-60%)
- needle=1529: 2.43 ms +- 1.14 ms -> 967 us +- 841 us: 2.51x faster (-60%)
- needle=2: 660 ns +- 1060 ns -> 263 ns +- 627 ns: 2.50x faster (-60%)
- needle=11621: 2.40 ms +- 1.02 ms -> 960 us +- 931 us: 2.50x faster (-60%)
- needle=679: 2.48 ms +- 1.11 ms -> 1.03 ms +- 0.95 ms: 2.42x faster (-59%)
- needle=1019: 2.48 ms +- 1.28 ms -> 1.08 ms +- 0.99 ms: 2.30x faster (-56%)
- needle=2294: 2.47 ms +- 1.17 ms -> 1.07 ms +- 0.93 ms: 2.30x faster (-56%)
- needle=452: 2.14 ms +- 0.99 ms -> 963 us +- 956 us: 2.23x faster (-55%)
- needle=17432: 2.24 ms +- 1.00 ms -> 1.12 ms +- 0.90 ms: 2.00x faster (-50%)
- needle=7747: 2.24 ms +- 1.18 ms -> 1.14 ms +- 0.99 ms: 1.97x faster (-49%)
- needle=26149: 2.29 ms +- 0.85 ms -> 1.33 ms +- 0.98 ms: 1.72x faster (-42%)
- needle=58837: 2.24 ms +- 1.00 ms -> 1.35 ms +- 0.94 ms: 1.66x faster (-40%)
- needle=88256: 2.40 ms +- 0.97 ms -> 1.52 ms +- 0.88 ms: 1.58x faster (-37%)
- needle=39224: 2.20 ms +- 1.00 ms -> 1.50 ms +- 0.92 ms: 1.46x faster (-32%)
- needle=88: 584 us +- 271 us -> 462 us +- 324 us: 1.26x faster (-21%)
- needle=1: 24.7 ns +- 2.4 ns -> 23.8 ns +- 2.1 ns: 1.04x faster (-4%)
Benchmark hidden because not significant (10): needle=3, needle=4, needle=5, needle=6, needle=10, needle=25, needle=38, needle=58, needle=133, needle=200

Geometric mean: 0.70 (faster)


22:12:11 vstinner@apu$ PYTHONPATH=~/myprojects/pyperf/ python3 -m pyperf compare_to ref.json pr22679.json --table -G
+----------------+---------+------------------------------+
| Benchmark      | ref     | pr22679                      |
+================+=========+==============================+
| needle=8       | 816 us  | 1.13 ms: 1.38x slower (+38%) |
+----------------+---------+------------------------------+
| needle=16      | 579 us  | 780 us: 1.35x slower (+35%)  |
+----------------+---------+------------------------------+
| needle=7       | 865 us  | 1.15 ms: 1.33x slower (+33%) |
+----------------+---------+------------------------------+
| needle=9       | 827 us  | 1.04 ms: 1.26x slower (+26%) |
+----------------+---------+------------------------------+
| needle=1       | 24.7 ns | 23.8 ns: 1.04x faster (-4%)  |
+----------------+---------+------------------------------+
| needle=88      | 584 us  | 462 us: 1.26x faster (-21%)  |
+----------------+---------+------------------------------+
| needle=39224   | 2.20 ms | 1.50 ms: 1.46x faster (-32%) |
+----------------+---------+------------------------------+
| needle=88256   | 2.40 ms | 1.52 ms: 1.58x faster (-37%) |
+----------------+---------+------------------------------+
| needle=58837   | 2.24 ms | 1.35 ms: 1.66x faster (-40%) |
+----------------+---------+------------------------------+
| needle=26149   | 2.29 ms | 1.33 ms: 1.72x faster (-42%) |
+----------------+---------+------------------------------+
| needle=7747    | 2.24 ms | 1.14 ms: 1.97x faster (-49%) |
+----------------+---------+------------------------------+
| needle=17432   | 2.24 ms | 1.12 ms: 2.00x faster (-50%) |
+----------------+---------+------------------------------+
| needle=452     | 2.14 ms | 963 us: 2.23x faster (-55%)  |
+----------------+---------+------------------------------+
| needle=2294    | 2.47 ms | 1.07 ms: 2.30x faster (-56%) |
+----------------+---------+------------------------------+
| needle=1019    | 2.48 ms | 1.08 ms: 2.30x faster (-56%) |
+----------------+---------+------------------------------+
| needle=679     | 2.48 ms | 1.03 ms: 2.42x faster (-59%) |
+----------------+---------+------------------------------+
| needle=11621   | 2.40 ms | 960 us: 2.50x faster (-60%)  |
+----------------+---------+------------------------------+
| needle=2       | 660 ns  | 263 ns: 2.50x faster (-60%)  |
+----------------+---------+------------------------------+
| needle=1529    | 2.43 ms | 967 us: 2.51x faster (-60%)  |
+----------------+---------+------------------------------+
| needle=5164    | 2.51 ms | 991 us: 2.53x faster (-60%)  |
+----------------+---------+------------------------------+
| needle=301     | 1.72 ms | 652 us: 2.64x faster (-62%)  |
+----------------+---------+------------------------------+
| needle=3442    | 2.24 ms | 846 us: 2.65x faster (-62%)  |
+----------------+---------+------------------------------+
| Geometric mean | (ref)   | 0.70 (faster)                |
+----------------+---------+------------------------------+

Not significant (10): needle=5; needle=4; needle=38; needle=10; needle=6; needle=200; needle=58; needle=25; needle=3; needle=133
History
Date User Action Args
2020-10-13 21:04:17vstinnersetrecipients: + vstinner, tim.peters, pmpp, serhiy.storchaka, josh.r, ammar2, corona10, Dennis Sweeney, Zeturic
2020-10-13 21:04:17vstinnersetmessageid: <1602623057.51.0.41332997237.issue41972@roundup.psfhosted.org>
2020-10-13 21:04:17vstinnerlinkissue41972 messages
2020-10-13 21:04:17vstinnercreate