This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Race condition in WeakKeyDictionary/WeakKeyDictionary
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: colesbury
Priority: normal Keywords: patch

Created on 2021-11-15 19:02 by colesbury, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
issue45809-repro.patch colesbury, 2021-11-15 19:05 Patch to make reproduction easier
Messages (2)
msg406357 - (view) Author: Sam Gross (colesbury) * (Python triager) Date: 2021-11-15 19:02
The issue described issue7105 (and maybe issue7060) still exists due to a race condition in WeakKeyDictionary. This shows up as test failure that looks like:

  test test_weakref failed -- Traceback (most recent call last):
    File "Lib/test/test_weakref.py", line 1960, in test_threaded_weak_value_dict_deepcopy
      self.check_threaded_weak_dict_copy(weakref.WeakValueDictionary, True)
    File "Lib/test/test_weakref.py", line 1940, in check_threaded_weak_dict_copy
      raise exc[0]
    File "Lib/test/test_weakref.py", line 1897, in dict_copy
      _ = copy.deepcopy(d)
    File "Lib/copy.py", line 153, in deepcopy
      y = copier(memo)
    File "Lib/weakref.py", line 189, in __deepcopy__
      for key, wr in self.data.items():
  RuntimeError: dictionary changed size during iteration

The cause is that the check of "self._iterating" and the call to "_atomic_removal" are not performed atomically together. By the time _atomic_removal() is called, an iteration might have already started.

https://github.com/python/cpython/blob/ec382fac0db6d9159c2d3496a70b7a605545957e/Lib/weakref.py#L109-L114
msg406358 - (view) Author: Sam Gross (colesbury) * (Python triager) Date: 2021-11-15 19:05
The attached patch (issue45809-repro.patch) introduces artificial delays to make reproduction of the underlying issue easier.

To reproduce the issue:
  
  patch -p1 < issue45809-repro.patch
  ./python -m test test_weakref -m test_threaded_weak_value_dict_deepcopy -v
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89967
2021-11-15 19:05:22colesburysetfiles: + issue45809-repro.patch
keywords: + patch
messages: + msg406358
2021-11-15 19:02:38colesburycreate