classification
Title: Overallocation is never disabled in _PyBytes_FormatEx()
Type: performance Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: inada.naoki, izbyshev, miss-islington, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-08-19 18:51 by izbyshev, last changed 2018-08-23 08:54 by izbyshev. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8826 merged izbyshev, 2018-08-19 18:58
PR 8865 merged miss-islington, 2018-08-23 07:51
PR 8866 merged miss-islington, 2018-08-23 07:51
Messages (7)
msg323763 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-08-19 18:51
The condition for disabling overallocation at 225b055/Objects/bytesobject.c:822 is always false.

Reported by Svace static analyzer.
msg323924 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 07:05
bytes % args has been optimized using the _PyBytesWriter API in bpo-25349: commit fa7762ec066aa3632a25b6a52bb7597b8f17c2f3.

I converted my old benchmark from bpo-25349 to my new perf module:
https://github.com/vstinner/pymicrobench/blob/master/bench_bytes_format_int.py

See also my article on _PyBytesWriter API: https://vstinner.github.io/pybyteswriter.html
msg323925 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 07:50
I ran a microbenchmark on PR 8826:

./python -m perf timeit -s 'fmt=b"%s%s"; args=(b"1", b"x"*(1024*1024))' 'fmt % args' -v -o patch.json

$ python3 -m perf compare_to ref.json patch.json
Mean +- std dev: [ref] 325 us +- 7 us -> [patch] 36.2 us +- 1.1 us: 8.98x faster (-89%)

Ooops, yeah, it's obvious that there was a missed optimization. It's now way faster ;-)
msg323926 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-23 07:51
New changeset ccd99752675042bd5f67d332c5b0ed85ba1f2da3 by Victor Stinner (Alexey Izbyshev) in branch 'master':
bpo-34436: Fix check that disables overallocation for the last fmt specifier (GH-8826)
https://github.com/python/cpython/commit/ccd99752675042bd5f67d332c5b0ed85ba1f2da3
msg323927 - (view) Author: miss-islington (miss-islington) Date: 2018-08-23 08:03
New changeset 042082692b7fab7361a8c9d0fb792532bb77e293 by Miss Islington (bot) in branch '3.7':
bpo-34436: Fix check that disables overallocation for the last fmt specifier (GH-8826)
https://github.com/python/cpython/commit/042082692b7fab7361a8c9d0fb792532bb77e293
msg323928 - (view) Author: miss-islington (miss-islington) Date: 2018-08-23 08:15
New changeset 945771b3c7a08f0b2b875905d517a1566477bfaf by Miss Islington (bot) in branch '3.6':
bpo-34436: Fix check that disables overallocation for the last fmt specifier (GH-8826)
https://github.com/python/cpython/commit/945771b3c7a08f0b2b875905d517a1566477bfaf
msg323931 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-08-23 08:54
Thank you for benchmarking and merging, Victor!
History
Date User Action Args
2018-08-23 08:54:55izbyshevsetstatus: open -> closed
resolution: fixed
messages: + msg323931

stage: patch review -> resolved
2018-08-23 08:15:06miss-islingtonsetmessages: + msg323928
2018-08-23 08:03:05miss-islingtonsetnosy: + miss-islington
messages: + msg323927
2018-08-23 07:51:34miss-islingtonsetpull_requests: + pull_request8340
2018-08-23 07:51:24miss-islingtonsetpull_requests: + pull_request8339
2018-08-23 07:51:03vstinnersetmessages: + msg323926
2018-08-23 07:50:05vstinnersetmessages: + msg323925
2018-08-23 07:05:22vstinnersetmessages: + msg323924
2018-08-19 18:58:51izbyshevsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8305
2018-08-19 18:51:43izbyshevcreate