classification
Title: dictkeysobject: Add maximum iteration check for .values() and .items()
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: inada.naoki, pitrou, rhettinger, serhiy.storchaka, thomas.perl
Priority: normal Keywords: patch

Created on 2019-03-29 14:16 by thomas.perl, last changed 2019-04-02 09:30 by inada.naoki. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12619 merged thomas.perl, 2019-03-29 14:19
Messages (4)
msg339115 - (view) Author: Thomas Perl (thomas.perl) * Date: 2019-03-29 14:16
On top of issue 36452, I noticed some other corner cases that are still not handled. For one, the patch (Github PR 12596) only handles iterating over keys, but not iterating over values or items:

======
a = {0: 0}
it = iter(a.values())
print('Length hint:', it.__length_hint__())
print(next(it))
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 99
print(next(it))
print('Length hint:', it.__length_hint__())
======

Replace a.values() there with a.items() -- same issue. Note that PR 12596 fixes the a.keys() case (same as iterating over "a" directly).

Applying the "di->len == 0" check in dictiter_iternextvalue() and dictiter_iternextitem() would fix those two cases above, but would still not fix the following case:

======
a = {0: 'a', 1: 'b', 2: 'c'}

it = iter(a)
i = next(it)
print('got first:', i)
del a[1]
a[1] = 'd'
i = next(it)
print('got second:', i)
i = next(it)
print('got third:', i)

try:
    i = next(it)
    raise RuntimeError(f'got fourth: {i}')
except StopIteration:
    print('stop iteration')
======

The reason for this is that the iteration count (3 in this case) isn't modified, but the dict's keys are still changed, and the iteration order is as follows:

======
got first: 0
got second: 2
got third: 1
stop iteration
======

Note that the key 1 there is first deleted and then set.

I'll add a Github PR that tries to solve these corner cases too by tracking dict keys modification.
msg339117 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-29 14:37
This is a duplicate of issue6017, issue19332 and issue29420.
msg339253 - (view) Author: Thomas Perl (thomas.perl) * Date: 2019-03-31 09:54
Repurposing this as per:

https://github.com/python/cpython/pull/12619#issuecomment-478076996
msg339324 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-04-02 09:30
New changeset b8311cf5e5d72f8a8aa688b7da1760d6a74a4d72 by Inada Naoki (Thomas Perl) in branch 'master':
bpo-36473: add maximum iteration check for dict .values() and .items() (GH-12619)
https://github.com/python/cpython/commit/b8311cf5e5d72f8a8aa688b7da1760d6a74a4d72
History
Date User Action Args
2019-04-02 09:30:30inada.naokisetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-04-02 09:30:12inada.naokisetmessages: + msg339324
2019-03-31 09:54:37thomas.perlsetmessages: + msg339253
title: Detect all dictionary changes during iteration -> dictkeysobject: Add maximum iteration check for .values() and .items()
2019-03-29 14:37:11serhiy.storchakasetnosy: + rhettinger, serhiy.storchaka, pitrou
messages: + msg339117
2019-03-29 14:36:08xtreaksetnosy: + inada.naoki
2019-03-29 14:19:55thomas.perlsetkeywords: + patch
stage: patch review
pull_requests: + pull_request12554
2019-03-29 14:16:18thomas.perlcreate