classification
Title: Double deallocation on iterator exhausting
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: python-dev, rhettinger, scorp, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-03-06 18:26 by serhiy.storchaka, last changed 2016-03-31 12:17 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
free_after_iterating.patch serhiy.storchaka, 2016-03-06 18:26 review
free_after_iterating_2.patch serhiy.storchaka, 2016-03-06 20:23 review
free_after_iterating_3.patch serhiy.storchaka, 2016-03-08 11:26 review
Messages (8)
msg261263 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-06 18:26
Following example causes double deallocation of a sequence and crashing.

class A(list):
    def __del__(self):
        next(it)

it = iter(A())
next(it)

The same is for subclass of tuple, str, bytes and bytearray.

Proposed patch fixes this issue.
msg261270 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-06 20:23
Updated patch fixes also set, dict and os.scandir() iterator.

May be sqlite3 cursor needs a fix, but it is too complicated.
msg261343 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-08 11:01
TODO: After resolving issue26492 add the test for array.
msg261348 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-08 11:26
Fixed test for OrderedDict. Tests for list and tuple are moved up to seq_tests.py.
msg261420 - (view) Author: Filipp Andjelo (scorp) Date: 2016-03-09 12:29
Hi Serhiy,

I tried the short example you gave, but it doesn't crash. I'm getting:

Exception ignored in: <bound method A.__del__ of []>
Traceback (most recent call last):
  File "./test.py", line 5, in __del__
    next(it)
StopIteration
Exception ignored in: <bound method A.__del__ of []>
Traceback (most recent call last):
  File "./test.py", line 5, in __del__
    next(it)
StopIteration
Traceback (most recent call last):
  File "./test.py", line 8, in <module>
    next(it)
StopIteration

Am I missing something?
msg261426 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-09 14:06
It crashes in debug build.
msg261734 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-03-14 08:58
Could anyone please look at the patch? I'm not sure about organizing tests.
msg262675 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-03-30 17:44
New changeset 905b5944119c by Serhiy Storchaka in branch '3.5':
Issue #26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/905b5944119c

New changeset 73ce47d4a7b2 by Serhiy Storchaka in branch 'default':
Issue #26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/73ce47d4a7b2

New changeset cff06d875678 by Serhiy Storchaka in branch '2.7':
Issue #26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/cff06d875678
History
Date User Action Args
2016-03-31 12:17:47serhiy.storchakasetstatus: open -> closed
assignee: serhiy.storchaka
resolution: fixed
stage: patch review -> resolved
2016-03-30 17:44:12python-devsetnosy: + python-dev
messages: + msg262675
2016-03-14 08:58:40serhiy.storchakasetmessages: + msg261734
2016-03-09 14:06:55serhiy.storchakasetmessages: + msg261426
2016-03-09 12:29:14scorpsetnosy: + scorp
messages: + msg261420
2016-03-08 11:26:29serhiy.storchakasetfiles: + free_after_iterating_3.patch

messages: + msg261348
2016-03-08 11:01:22serhiy.storchakasetmessages: + msg261343
2016-03-07 07:21:28serhiy.storchakalinkissue26496 dependencies
2016-03-06 20:23:25serhiy.storchakasetfiles: + free_after_iterating_2.patch

messages: + msg261270
2016-03-06 18:26:04serhiy.storchakacreate