This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Reduce deque repeat execution when maxlen exist and size is not 1
Type: enhancement Stage: resolved
Components: Extension Modules Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: louielu, rhettinger, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-02-23 16:39 by louielu, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 255 merged louielu, 2017-02-23 16:44
Messages (7)
msg288460 - (view) Author: Louie Lu (louielu) * Date: 2017-02-23 16:39
There is a XXX in v3.5.0 shows that need to dealing with deque maxlen setting case in deque_repeat.

Although there have common case for deque size 1 with maxlen, other size of deque with maxlen still using for-loop to extend the deque, without any detection.

Adding this fast break will reduce the execution speed when repeat deque with maxlen.

---
$ cat tests.py
from collections import deque
for _ in range(10:
    d = deque(maxlen=100_000)
    d.insert(0, 0)
    d.insert(0, 10)
    d * 10_000_000
$ time ./python_with_patch tests.py
$ time ./python tests.py
msg288462 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-23 16:56
Wouldn't be better to update the number of repeats before the loop rather than checking at every iteration?

if (deque->maxlen >= 0 && n * size > deque->maxlen)
    n = (deque->maxlen + size - 1) / size;
msg288463 - (view) Author: Louie Lu (louielu) * Date: 2017-02-23 17:01
Serhiy: yes, your advice is better than checking inside the loop.

I have updated this to the commit, thanks!
msg288496 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-02-23 23:49
The code looks fine.  Please change the comment to something like:

/* Reduce the number of repetitions when maxlen would be exceeded */
msg288501 - (view) Author: Louie Lu (louielu) * Date: 2017-02-24 02:30
Raymond: comment has changed, pushed on to GitHub.
msg288507 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-24 08:15
There is an opportunity of further optimisation. For example deque(range(100000), maxlen=100001) *= 2 copies 100000 elements while it is enough to copy just 1 element. But I don't know whether it is worth to optimise this.
msg290415 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-03-24 23:49
New changeset 357bad7101df196d7f4ea3dc6ed9a6436afb022c by Raymond Hettinger (Louie Lu) in branch 'master':
bpo-29634: Reduce deque repeat execution when maxlen exist and size is not 1 (#255) (#255)
https://github.com/python/cpython/commit/357bad7101df196d7f4ea3dc6ed9a6436afb022c
History
Date User Action Args
2022-04-11 14:58:43adminsetgithub: 73820
2017-03-24 23:49:14rhettingersetmessages: + msg290415
2017-02-24 08:15:54serhiy.storchakasetmessages: + msg288507
2017-02-24 04:00:41rhettingersetstatus: open -> closed
resolution: fixed
stage: resolved
2017-02-24 02:30:35louielusetmessages: + msg288501
2017-02-23 23:49:55rhettingersetmessages: + msg288496
2017-02-23 17:01:30louielusetmessages: + msg288463
2017-02-23 16:56:24serhiy.storchakasetassignee: rhettinger

messages: + msg288462
nosy: + rhettinger, serhiy.storchaka
2017-02-23 16:44:46louielusetpull_requests: + pull_request223
2017-02-23 16:39:23louielucreate