This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Race condition in
Type: behavior Stage: resolved
Components: IO Versions: Python 3.7, Python 3.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: barry, eli.bendersky, ethan.furman, gz, python-dev, simon.percivall
Priority: normal Keywords: patch

Created on 2017-01-05 10:54 by simon.percivall, last changed 2022-04-11 14:58 by admin. This issue is now closed.

File name Uploaded Description Edit
issue29167.stoneleaf.01.patch ethan.furman, 2017-01-23 18:19 review
Messages (8)
msg284724 - (view) Author: Simon Percivall (simon.percivall) Date: 2017-01-05 10:54
When called by `_create_pseudo_member_()`, the dictionary iteration of `_value2member_map` in `_decompose()` in may lead to a "RuntimeError: dictionary changed size during iteration". For me, it happened in `re.compile`.

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 233, in compile
    return _compile(pattern, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 301, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 562, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 866, in parse
    p.pattern.flags = fix_flags(str, p.pattern.flags)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 835, in fix_flags
    flags |= SRE_FLAG_UNICODE
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 794, in __or__
    result = self.__class__(self._value_ | self.__class__(other)._value_)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 291, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 533, in __new__
    return cls._missing_(value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 760, in _missing_
    new_member = cls._create_pseudo_member_(value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 769, in _create_pseudo_member_
    _, extra_flags = _decompose(cls, value)
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 849, in _decompose
    for v, m in flag._value2member_map_.items()
  File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/", line 848, in <listcomp>
    (m, v)
RuntimeError: dictionary changed size during iteration
msg284761 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-01-05 16:59
Simon, can you post the exact line of code that causes the error?  It would be useful for creating a test case and the couple things I have tried to duplicate the error have worked fine.
msg284928 - (view) Author: Simon Percivall (simon.percivall) Date: 2017-01-07 18:39
Run this a couple of times (it fails for me the first time, but it's a race, so YMMV):

import enum
from concurrent.futures import ThreadPoolExecutor

class MyEnum(enum.IntFlag):
    one = 1

with ThreadPoolExecutor() as executor:
    print(list(, range(1000))))

An easy fix would be:

diff --git a/Lib/ b/Lib/
index e79b0382eb..eca56ec3a7 100644
--- a/Lib/
+++ b/Lib/
@@ -846,7 +846,7 @@ def _decompose(flag, value):
         # check for named flags and powers-of-two flags
         flags_to_check = [
                 (m, v)
-                for v, m in flag._value2member_map_.items()
+                for v, m in list(flag._value2member_map_.items())
                 if is not None or _power_of_two(v)
     members = []
msg284939 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-01-07 21:56
Thanks.  I'll go through and audit all my dictionary iterations.
msg286105 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-01-23 18:19
Fixed the race condition for both the RuntimeError and for getting duplicate composite members.
msg286213 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-01-24 20:14
New changeset 95e184bd2d89 by Ethan Furman in branch '3.6':
closes issue29167: fix race condition in (Int)Flag

New changeset e6b98c270718 by Ethan Furman in branch 'default':
issue29167: fix race condition in (Int)Flag
msg337138 - (view) Author: Martin (gz) * Date: 2019-03-04 16:45
Our production system hit this issue using Python 3.6.7 once a few days ago, so presumably the race is still possible with the applied patch, just less likely?

RuntimeError: dictionary changed size during iteration
at _decompose (/usr/lib/python3.6/
at _create_pseudo_member_ (/usr/lib/python3.6/
at _missing_ (/usr/lib/python3.6/
at __new__ (/usr/lib/python3.6/
at __call__ (/usr/lib/python3.6/
at __or__ (/usr/lib/python3.6/
at create_urllib3_context (/usr/local/lib/python3.6/dist-packages/urllib3/util/
at connect (/usr/local/lib/python3.6/dist-packages/urllib3/
at _validate_conn (/usr/local/lib/python3.6/dist-packages/urllib3/
at _make_request (/usr/local/lib/python3.6/dist-packages/urllib3/
at urlopen (/usr/local/lib/python3.6/dist-packages/urllib3/
at send (/usr/local/lib/python3.6/dist-packages/requests/
at send (/usr/local/lib/python3.6/dist-packages/requests/
at request (/usr/local/lib/python3.6/dist-packages/requests/
at __call__ (/usr/local/lib/python3.6/dist-packages/google/auth/transport/
at _token_endpoint_request (/usr/local/lib/python3.6/dist-packages/google/oauth2/
at jwt_grant (/usr/local/lib/python3.6/dist-packages/google/oauth2/
at refresh (/usr/local/lib/python3.6/dist-packages/google/oauth2/
at before_request (/usr/local/lib/python3.6/dist-packages/google/auth/
at request (/usr/local/lib/python3.6/dist-packages/google/auth/transport/
at wait_and_retry (/usr/local/lib/python3.6/dist-packages/google/resumable_media/
at http_request (/usr/local/lib/python3.6/dist-packages/google/resumable_media/requests/
at consume (/usr/local/lib/python3.6/dist-packages/google/resumable_media/requests/
at _do_download (/usr/local/lib/python3.6/dist-packages/google/cloud/storage/
at download_to_file (/usr/local/lib/python3.6/dist-packages/google/cloud/storage/
msg357471 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2019-11-26 01:01
The latest patch from issue38045 should make race-conditions non-existent.
Date User Action Args
2022-04-11 14:58:41adminsetgithub: 73353
2019-11-26 01:01:22ethan.furmansetmessages: + msg357471
2019-03-04 16:45:57gzsetnosy: + gz
messages: + msg337138
2017-01-24 20:14:24python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg286213

resolution: fixed
stage: patch review -> resolved
2017-01-23 18:19:44ethan.furmansetfiles: + issue29167.stoneleaf.01.patch
keywords: + patch
messages: + msg286105

stage: test needed -> patch review
2017-01-07 21:56:28ethan.furmansetnosy: + barry, eli.bendersky
messages: + msg284939
2017-01-07 18:39:54simon.percivallsetmessages: + msg284928
2017-01-05 17:00:19ethan.furmansetassignee: ethan.furman
stage: test needed
2017-01-05 16:59:53ethan.furmansetmessages: + msg284761
2017-01-05 10:57:17serhiy.storchakasetnosy: + ethan.furman
type: crash -> behavior
components: + IO
2017-01-05 10:54:26simon.percivallcreate