classification
Title: 'dictionary changed size' error in test_multiprocessing
Type: Stage:
Components: Extension Modules Versions: Python 3.0
process
Status: closed Resolution: duplicate
Dependencies: Superseder: weak dict iterators are fragile because of unpredictable GC runs
View: 7105
Assigned To: jnoller Nosy List: donmez, jnoller, mark.dickinson, pitrou
Priority: normal Keywords:

Created on 2008-08-17 13:42 by mark.dickinson, last changed 2009-10-24 12:55 by pitrou. This issue is now closed.

Messages (8)
msg71274 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-08-17 13:42
Here's a report from Ismail Donmez (cartman), extracted from the
issue 3419 discussion in an effort to avoid putting multiple
problems under one tracker issue.

[cartman]

With trunk when running test_multiprocessing in a tight loop I saw 
another problem:

test_multiprocessing
Process Process-61:
Traceback (most recent call last):
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/process.py", 
line 229, in _bootstrap
Process Process-60:
Traceback (most recent call last):
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/process.py", 
line 229, in _bootstrap
Process Process-62:
Traceback (most recent call last):
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/process.py", 
line 229, in _bootstrap
    util._run_after_forkers()
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/util.py", line 
138, in _run_after_forkers
    util._run_after_forkers()
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/util.py", line 
138, in _run_after_forkers
    util._run_after_forkers()
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/util.py", line 
138, in _run_after_forkers
    items = list(_afterfork_registry.items())
    items = list(_afterfork_registry.items())
  File "/Users/cartman/Sources/py3k/Lib/weakref.py", line 103, in items
  File "/Users/cartman/Sources/py3k/Lib/weakref.py", line 103, in items
    items = list(_afterfork_registry.items())
  File "/Users/cartman/Sources/py3k/Lib/weakref.py", line 103, in items
    for key, wr in self.data.items():
RuntimeError: dictionary changed size during iteration
    for key, wr in self.data.items():
RuntimeError: dictionary changed size during iteration
    for key, wr in self.data.items():
RuntimeError: dictionary changed size during iteration
msg71343 - (view) Author: Ismail Donmez (donmez) * Date: 2008-08-18 15:52
I don't know if this is reproducible in 2.6 but I can reproduce on py3k 
branch, and also thanks for creating the bug!
msg71346 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-18 16:12
Mmmh, the problem with the list(X.items()) idiom is that it's
thread-safe only if X.items() is implemented in C. Otherwise X can be
mutated if there is a thread-switch while executing bytecode in X.items().

In weakref.py (line 103), by replacing:
    for key, wr in self.data.items():
with:
    for key, wr in list(self.data.items()):

This particular error should disappear.

But this doesn't say why the dictionary is mutated at all. Does
multiprocessing (or at least that particular test) launch several
threads in a given process? Otherwise there may be something fishy going on.
msg71366 - (view) Author: Ismail Donmez (donmez) * Date: 2008-08-18 19:37
py3k branch gives another error now, when running test_multiprocessing 
in a tight loop:

test test_multiprocessing failed -- Traceback (most recent call last):
  File "/Users/cartman/Sources/py3k/Lib/test/test_multiprocessing.py", 
line 1163, in test_remote
    queue = manager2.get_queue()
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 641, in temp
    authkey=self._authkey, exposed=exp
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 893, in AutoProxy
    incref=incref)
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 702, in __init__
    self._incref()
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 749, in _incref
    dispatch(conn, None, 'incref', (self._id,))
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 85, in dispatch
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
------------------------------------------------------------------------
---
Traceback (most recent call last):
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 187, in handle_request
    result = func(c, *args, **kwds)
  File "/Users/cartman/Sources/py3k/Lib/multiprocessing/managers.py", 
line 403, in incref
    self.id_to_refcount[ident] += 1
KeyError: '7389d0'
msg71367 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-08-18 19:40
Ismail, that's the incref bug in issue 3419
msg71368 - (view) Author: Ismail Donmez (donmez) * Date: 2008-08-18 19:42
Ah cool, we might be at the end of multiprocessing problems then I guess 
:-)
msg71370 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-18 19:50
Le lundi 18 août 2008 à 19:42 +0000, Ismail Donmez a écrit :
> Ismail Donmez <ismail@namtrac.org> added the comment:
> 
> Ah cool, we might be at the end of multiprocessing problems then I guess 
> :-)

Well, not really, it should be diagnosed why the dictionary mutates in
the first place. This dictionary is apparently processed in a child
process just after a fork() occurred, so logically there shouldn't be
several threads running.
Unless processing the dictionary itself can create some new threads?
msg94411 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-10-24 12:55
Actually, it's probably a duplicate of #7105.
History
Date User Action Args
2009-10-24 12:55:46pitrousetstatus: open -> closed
resolution: duplicate
superseder: weak dict iterators are fragile because of unpredictable GC runs
messages: + msg94411
2009-01-22 19:54:54jnollersetpriority: normal
2008-08-18 19:50:35pitrousetmessages: + msg71370
2008-08-18 19:42:23donmezsetmessages: + msg71368
2008-08-18 19:40:50jnollersetmessages: + msg71367
2008-08-18 19:37:51donmezsetmessages: + msg71366
2008-08-18 16:12:32pitrousetnosy: + pitrou
messages: + msg71346
2008-08-18 15:52:00donmezsetmessages: + msg71343
versions: + Python 3.0, - Python 2.6
2008-08-17 13:42:07mark.dickinsoncreate