Alright, here's what's going on.
When the main thread exits, it triggers the interpreter shutdown, which clears all the tstates in PyInterpreterState_Clear():
"""
void
PyInterpreterState_Clear(PyInterpreterState *interp)
{
    PyThreadState *p;
    HEAD_LOCK();
    for (p = interp->tstate_head; p != NULL; p = p->next)
        PyThreadState_Clear(p);
"""

PyThreadState_Clear() clears the TLS dict:
"""
void
PyThreadState_Clear(PyThreadState *tstate)
{
    if (Py_VerboseFlag && tstate->frame != NULL)
        fprintf(stderr,
          "PyThreadState_Clear: warning: thread still has a frame\n");

    Py_CLEAR(tstate->frame);

    Py_CLEAR(tstate->dict);
"""

This deallocation of the TLS dict But when the TLS object is deallocated, if it releases the GIL, this can make other threads runnable, while the interpreter is shutting down (and the tstate are in an unusable state), so all  bets are off. Note that this can only happen if there are daemon threads, which is the case in your testcase.

Basically, the problem is that arbitrary code can be run while the interpreter is shutting down because of the TLS deallocation.

I'm not sure about how to handle it, but one possibility to limit such problems would be to not deallocate the tstate if a thread is currently still active:

"""
diff --git a/Python/pystate.c b/Python/pystate.c
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -230,9 +230,12 @@
 void
 PyThreadState_Clear(PyThreadState *tstate)
 {
-    if (Py_VerboseFlag && tstate->frame != NULL)
-        fprintf(stderr,
-          "PyThreadState_Clear: warning: thread still has a frame\n");
+    if (tstate->frame != NULL) {
+        if (Py_VerboseFlag)
+            fprintf(stderr,
+                    "PyThreadState_Clear: warning: thread still has a frame\n");
+        return;
+    }
 
     Py_CLEAR(tstate->frame);
 
"""

But this would leak to memory leak in some cases...