classification
Title: weak dict iterators are fragile because of unpredictable GC runs
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.7, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, ajaksu2, benjamin.peterson, dcjim, elachuni, gvanrossum, jon, mark.dickinson, pitrou, qelan, tseaver, vdupras
Priority: normal Keywords: patch

Created on 2009-10-11 17:29 by pitrou, last changed 2011-08-18 18:31 by qelan.

Files
File name Uploaded Description Edit
weakiter3.patch pitrou, 2009-10-13 17:24
Messages (8)
msg93863 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-11 17:29
As mentioned in issue7060, weak dict iterators are easily broken by
cyclic garbage collection changing the size of the underlying dict storage:

  File "/home/rdmurray/python/py3k/Lib/weakref.py", line 121, in items
    for wr in self.data.values():
RuntimeError: dictionary changed size during iteration

One possible solution is to delay all removals until all iterators are
done. Here is a context manager-based solution.
msg93865 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2009-10-11 17:57
> delay all removals until all iterators are done

+1
msg93912 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-12 23:50
This new patch makes it possible to mutate the dict without messing with
the delayed removal when an iterator exists.
msg93934 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-13 17:24
It occurs weaksets have the same problem, here is a new patch fixing
them as well.
msg97392 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-01-08 02:11
LGTM
msg97425 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-08 17:57
Committed in r77365 (py3k) and r77366 (3.1). Thank you.
msg111259 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-07-23 09:57
The issue still exists in 2.6 and 2.7.

Closing issue 839159 as a duplicate of this one.
msg142392 - (view) Author: Matthew Schwickerath (qelan) Date: 2011-08-18 18:31
Any plans on actually patching this in 2.7 any time soon?  This is affecting our software and hanging it on random occasions.
History
Date User Action Args
2011-08-18 18:31:12qelansetnosy: + qelan
messages: + msg142392
2010-07-23 09:58:05mark.dickinsonsetnosy: + dcjim, tseaver, ajaksu2, vdupras, elachuni, BreamoreBoy
2010-07-23 09:57:23mark.dickinsonsetstatus: closed -> open

versions: + Python 2.6, Python 2.7, - Python 3.1, Python 3.2
nosy: + mark.dickinson

messages: + msg111259
resolution: fixed ->
stage: committed/rejected ->
2010-07-23 09:56:04mark.dickinsonlinkissue839159 superseder
2010-01-08 17:57:03pitrousetstatus: open -> closed
resolution: fixed
messages: + msg97425

stage: patch review -> committed/rejected
2010-01-08 02:11:35benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg97392
2009-11-13 17:39:30pitrousetfiles: - weakiter2.patch
2009-11-13 17:39:28pitrousetfiles: - weakiter.patch
2009-10-24 12:55:46pitroulinkissue3578 superseder
2009-10-13 17:24:59pitrousetfiles: + weakiter3.patch

messages: + msg93934
2009-10-12 23:51:00pitrousetfiles: + weakiter2.patch

messages: + msg93912
2009-10-11 17:57:27gvanrossumsetnosy: + gvanrossum
messages: + msg93865
2009-10-11 17:36:34jonsetnosy: + jon
2009-10-11 17:29:50pitroulinkissue7060 dependencies
2009-10-11 17:29:21pitroucreate