> So if you are going to eliminate daemon threads (even if only in sub interpreters at this point), you are going to have to introduce a way to register something similar to an atexit callback which would be invoked before waiting on non daemon threads, so an attempt can be made to notify them that they need to shutdown.

I'm not 100% certain if it fully covers the above use cases, but I recently added a fairly minimal internal `threading._register_atexit()` function that works similar to atexit: it schedules the callbacks to be invoked just before all non-daemon threads are joined, rather than upon interpreter finalization. The primary use case was to remove the reliance of daemon threads in concurrent.futures, for ThreadPoolExecutor and ProcessPoolExecutor. Their management threads relied fully upon daemon threads and `atexit.register()`, but the above provided an alternative means to accomplish the same goal; and also has the benefit of a more predictable shutdown process. See https://github.com/python/cpython/pull/19149 for details.

Should we consider making `threading._register_atexit()` public, and if we did, would it provide a reasonable transition for users that were previously relying upon daemon threads if they were to be deprecated in subinterpreters? If it were to be made public, there may also be a need to be able to unregister the threading atexit callbacks, similar to `atexit.unregister()` (that should be rather straightforward though).

(I added Antoine Pitrou to the nosy list, to see if he has an opinion on any of this as the maintainer of the threading module.)