Well, this large problem sounds very complex and is made of multiple small issues. Let's start with something simple: revert my commit 066e5b1a917ec2134e8997d2cadd815724314252.

When I wrote it, I expected that nobody was spawning daemon threads in subinterpreters. I was wrong.

Once PR 19456 will be merged (allow again to spawn daemon threads in subinterpreters), we can start to investigate every single issue listed by Graham, like atexit callbacks which are called too late.