This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: KeyError at exit after 'import threading' in other thread
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10, Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Laurent.Mazuel, Rhamphoryncus, alexis, amaury.forgeotdarc, brian.curtin, bronger, cmcqueen1975, cwalther, eric.snow, gregory.p.smith, guettli, hasenpfeffer, iritkatriel, miss-islington, pitrou, vstinner
Priority: normal Keywords: patch

Created on 2006-11-14 14:02 by cwalther, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit nobody, 2006-11-15 07:00 minimal reproducing example amaury.forgeotdarc, 2008-01-17 20:04
Pull Requests
URL Status Linked Edit
PR 28549 merged vstinner, 2021-09-24 15:30
PR 28588 merged miss-islington, 2021-09-27 21:09
PR 28589 merged vstinner, 2021-09-27 21:12
Messages (26)
msg30544 - (view) Author: Christian Walther (cwalther) Date: 2006-11-14 14:02
Python 2.4.3 on Windows 2000, though the code in
question seems unchanged in current SVN (r46919).

I'm using Python embedded in a multithreaded C++
application. When 'import threading' is first done in
some Python script that runs in thread A, I get the
following exception when a different thread B calls

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python24\lib\", line 24, in
    func(*targs, **kargs)
  File "C:\Python24\lib\", line 636, in
  File "C:\Python24\lib\", line 522, in
    del _active[_get_ident()]
KeyError: 680
Error in sys.exitfunc:
Traceback (most recent call last):
  File "C:\Python24\lib\", line 24, in
    func(*targs, **kargs)
  File "C:\Python24\lib\", line 636, in
  File "C:\Python24\lib\", line 522, in
    del _active[_get_ident()]
KeyError: 680

The reason seems to be that the threading module uses
the thread ID of the calling thread as a key to store
its _MainThread instance on initialization, and again
the thread ID of the calling thread to delete it in its
exit function. If these two threads are not the same,
the described KeyError occurs.

I didn't study this in all detail, but it seems to me
that threading.Thread.__delete() does the wrong thing.
By doing 'del _active[_get_ident()]', it removes the
instance for the calling thread from the _active
dictionary. What it should be doing is removing *self*
from that dictionary. Is that correct?
msg30545 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2006-11-14 21:59
Well, I don't think you should be calling Py_Finalize() from the non-main thread.  That just seems unsafe to me.

Regardless, though, could you write up some quick Python code that triggers this?
msg30546 - (view) Author: Christian Walther (cwalther) Date: 2006-11-15 07:02
I'm not calling Py_Finalize from a non-main thread. What I called "thread B" is the main thread. It's the script that first imports the threading module that runs in a non-main thread (and running user-defined scripts in non-main threads is hopefully not unsafe, or there wouldn't be much point in supporting multithreading at all in Python).

It didn't even occur to me that this could be reproduced in pure Python code, so I didn't include an example in my original post. Of course, it can - see attachment. Tested on Python 2.3.5 on Mac OS X.
msg30547 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2006-11-15 18:51
Thanks for the test code.  I have no clue when I or anyone else will get to this, but the report and testing code is appreciated.
msg60013 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-01-17 01:47 doesn't fail for me on trunk (linux), anyone else?

the output I get is always:

Main thread ID: -134346528
Secondary thread ID: -135349328
Exception KeyError: KeyError(-134346528,) in <module 'threading' from
'/home/gps/oss/python/trunk/Lib/threading.pyc'> ignored
msg60035 - (view) Author: Christian Walther (cwalther) Date: 2008-01-17 17:13
I'm not sure what you mean by "doesn't fail" - from the output you quote, 
I'd say that it does fail. It's in fact the same output as I get right now 
with Python 2.5.1 on Mac OS X.

Would you classify that KeyError as expected behavior?
msg60041 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-01-17 18:14
gah, sorry i misread the report.  you are correct.
msg60050 - (view) Author: Adam Olsen (Rhamphoryncus) Date: 2008-01-17 18:54
Is the bug avoided if you import threading first and use it instead of
thread?  I'd like to see thread removed in 3.0 (renamed to _thread or
the like.)
msg60052 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-01-17 20:04
If a python daemon thread is still running when the interpreter exits,
it is likely to fail in random ways.

Here is another example, which does not use imports.
I run the script many times, with latest version in trunk, on Windows
XP, debug build.

In the majority of runs, I get an error message:
Exception in thread Thread-1 (most likely raised during interpreter
Other tests seem to show that all modules are emptied by the cleanup
process, but the thread insists to get "time.sleep".

And more interestingly, about every 50 runs, the process segfaults...
I suspect that this is a problem similar to
msg60083 - (view) Author: Christian Walther (cwalther) Date: 2008-01-18 08:54
> Is the bug avoided if you import threading first and use it instead of thread?

Yes. The bug happens when the (first) import of threading and the call to Py_Finalize() 
happen in different threads. To reproduce the problem in pure Python, I therefore have to 
use thread instead of threading to create the secondary thread. (In the C++ application, 
it's created on the C++ side.)

Has anyone checked if the solution I propose in the first post makes sense?
msg97989 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-01-18 04:44
FWIW, Amaury's example runs without error on trunk and py3k (OS X 10.5).

2.6 prints the following:
"Exception in thread Thread-1 (most likely raised during interpreter shutdown)"

3.1 seg faults
msg98031 - (view) Author: Christian Walther (cwalther) Date: 2010-01-18 20:40
I have the impression we're tracking two completely unrelated problems in this tracker item.

As to "needs patch" regarding my problem: Here's the solution I proposed in my original post in patch form - I'm just not sure if it is correct. I don't recommend applying this until someone who is familiar with the workings of the threading module has confirmed that removing self from _active is the right thing to do (and that what I'm doing is the accepted pythonic way of removing a dictionary entry by value).

Index: Lib/
--- Lib/    (revision 77598)
+++ Lib/    (working copy)
@@ -611,7 +611,11 @@
             with _active_limbo_lock:
-                del _active[_get_ident()]
+                for k, v in _active.iteritems():
+                    if v is self: break
+                else:
+                    assert False, "thread instance not found in _active"
+                del _active[k]
                 # There must not be any python code between the previous line
                 # and after the lock is released.  Otherwise a tracing function
                 # could try to acquire the lock again in the same thread, (in
msg101890 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-29 14:50
I think the fix to Christian's issue is just:

Index: Lib/
--- Lib/	(révision 79470)
+++ Lib/	(copie de travail)
@@ -579,7 +579,7 @@
                     # We don't call self.__delete() because it also
                     # grabs _active_limbo_lock.
-                    del _active[_get_ident()]
+                    del _active[self.__ident]
@@ -615,7 +615,7 @@
             with _active_limbo_lock:
-                del _active[_get_ident()]
+                del _active[self.__ident]
                 # There must not be any python code between the previous line
                 # and after the lock is released.  Otherwise a tracing function
                 # could try to acquire the lock again in the same thread, (in

Now we just need to add a test for it in test_threading.
And, yes, Amaury's test case looks like a different issue.
msg107903 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2010-06-16 02:54
From my limited experience using cx_Freeze 4.1.2 with Python 2.6.5, it seems that this issue is triggered in a cx_Frozen program simply by having `import threading` in the program. I'm not sure what cx_Freeze is doing that makes this issue show up.
msg107904 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2010-06-16 02:57
Sorry I should have said, I'm running on Windows 2000 SP4.
msg109088 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2010-07-02 02:59
A follow-on re the cx_Freeze issue: I looked at the source code, and found it doesn't seem to be doing any thread creation. But I found that in the initscripts/, there are the following lines:

    if sys.version_info[:2] >= (2, 5):
        module = sys.modules.get("threading")
        if module is not None:

If these lines are commented-out, then the error message at exit does not occur.
msg110202 - (view) Author: Laurent Mazuel (Laurent.Mazuel) Date: 2010-07-13 15:02
Another solution for cx-freeze problem:

Which can be added in ConsoleKeepPath.c for instance
msg133043 - (view) Author: (hasenpfeffer) Date: 2011-04-05 15:28
I encountered this issue recently in Python 3.2 and wanted to make some observations about it.

The real problem here is not the KeyError.  Though the suggested patches would fix the KeyError symptom, they do not fix the underlying issue.  The underlying issue is the threading module assumes that it is imported from the Python main thread.  When alien threads exist, the threading module may be imported (directly or indirectly) from a thread that is not the Python main thread, causing the wrong thread to be marked as the Python main thread.

The resulting problems of the wrong thread being marked as the Python main thread appear to be minor.  The KeyError at exit is one of them.  Another problem I encountered was with confusion in a threaded debugger that displayed my alien thread as the Python main thread, and the Python main thread as the alien thread.

In my project I can easily work around this by importing the threading module in a root package that is extremely likely to be imported from the Python main thread, causing the correct thread to be marked as the main thread.

Since I have a workaround for my project, in addition to the relatively minor issues that result from this behavior, I haven't spent any time looking into how to fix the underlying issue.  If I have the time, I'll suggest one.
msg194597 - (view) Author: Thomas Guettler (guettli) * Date: 2013-08-07 07:45
Only few people seem to use daemon threads. We do and see this problem often with Python 2.7.

How difficult is it to get this fixed for 2.7?

Is there a way to work around this problem?
msg358743 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-12-20 23:05
related: issue #39042 "Use the runtime's main thread ID in the threading module."
msg399693 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-08-16 22:17
The output is different now. 

I update the script for python 3:

import _thread as thread
import time

def start():
	print("Secondary thread ID:", thread.get_ident())
	import threading

print("Main thread ID:", thread.get_ident())
thread.start_new_thread(start, ())


and the output is:


Main thread ID: 4432801280
Secondary thread ID: 123145384259584
Exception ignored in: <module 'threading' from '/Users/iritkatriel/src/cpython-perf/Lib/'>
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-perf/Lib/", line 1512, in _shutdown
    assert tlock.locked()

msg402573 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-24 15:33
I proposed PR 28549 to fix this very old threading issue.

A C extension can spawn threads without using the threading module and then then run Python code which imports the threading module. In this case, threading._main_thread is the thread which imported first the threading module. My PR changes so threading._shutdown() simply ignores _main_thread when it happens.
msg402740 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-27 21:09
New changeset 95d31370829b7d729667588e0a9943217401ea5b by Victor Stinner in branch 'main':
bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549)
msg402744 - (view) Author: miss-islington (miss-islington) Date: 2021-09-27 21:39
New changeset 38c67738c64304928c68d5c2bd78bbb01d979b94 by Miss Islington (bot) in branch '3.10':
bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549)
msg402745 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-27 21:40
New changeset 94d19f606fa18a1c4d2faca1caf2f470a8ce6d46 by Victor Stinner in branch '3.9':
bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549) (GH-28589)
msg402746 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-27 21:41
Better late than never. I only took 15 years to fix this old bug :-D
Date User Action Args
2022-04-11 14:56:21adminsetgithub: 44231
2021-09-30 01:41:09vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-09-27 21:41:30vstinnersetmessages: + msg402746
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 2.7, Python 3.2
2021-09-27 21:40:26vstinnersetmessages: + msg402745
2021-09-27 21:39:50miss-islingtonsetmessages: + msg402744
2021-09-27 21:12:05vstinnersetpull_requests: + pull_request26971
2021-09-27 21:09:26miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26970
2021-09-27 21:09:12vstinnersetmessages: + msg402740
2021-09-24 15:33:38vstinnersetmessages: + msg402573
2021-09-24 15:30:26vstinnersetkeywords: + patch
nosy: + vstinner

pull_requests: + pull_request26932
stage: needs patch -> patch review
2021-08-16 22:17:30iritkatrielsetnosy: + iritkatriel
messages: + msg399693
2020-01-10 20:51:41brett.cannonsetnosy: - brett.cannon
2019-12-20 23:05:27eric.snowsetnosy: + eric.snow
messages: + msg358743
2013-08-07 07:45:33guettlisetnosy: + guettli
messages: + msg194597
2012-07-09 21:15:30alexissetnosy: + alexis
2011-04-05 15:28:51hasenpfeffersetnosy: + hasenpfeffer

messages: + msg133043
versions: + Python 2.7, Python 3.2, - Python 2.6, Python 3.1
2010-07-13 15:02:56Laurent.Mazuelsetnosy: + Laurent.Mazuel
messages: + msg110202
2010-07-02 02:59:09cmcqueen1975setmessages: + msg109088
2010-06-16 02:57:41cmcqueen1975setmessages: + msg107904
2010-06-16 02:54:58cmcqueen1975setnosy: + cmcqueen1975
messages: + msg107903
2010-03-29 14:50:42pitrousetnosy: + pitrou
messages: + msg101890
2010-03-29 08:20:55brongersetnosy: + bronger
2010-01-18 20:40:08cwalthersetmessages: + msg98031
2010-01-18 04:44:31brian.curtinsetversions: + Python 3.1, - Python 2.5, Python 2.4
nosy: + brian.curtin

messages: + msg97989

type: behavior
stage: needs patch
2009-02-17 17:20:49amaury.forgeotdarclinkissue1159425 superseder
2008-01-18 08:55:01cwalthersetmessages: + msg60083
2008-01-17 20:04:11amaury.forgeotdarcsetfiles: +
nosy: + amaury.forgeotdarc
messages: + msg60052
2008-01-17 18:54:46Rhamphoryncussetnosy: + Rhamphoryncus
messages: + msg60050
2008-01-17 18:14:06gregory.p.smithsetmessages: + msg60041
versions: + Python 2.6, Python 2.5, Python 2.4
2008-01-17 17:13:26cwalthersetmessages: + msg60035
2008-01-17 01:47:55gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg60013
2006-11-14 14:02:45cwalthercreate