> This means that ThreadPoolExecutor's atexit runs before mine,
> and since I never get a chance to cancel my tasks, it deadlocks.

(assuming we want to support long-running tasks here)

With all the above in mind, there are a few things that may help.

The first that comes to mind is to have the atexit handler call ThreadPoolExecutor.shutdown() for each instance.

So something like this:


def _python_exit():
    global _shutdown
    with _global_shutdown_lock:
        _shutdown = True
    for executor in list(_executors):
        executor.shutdown()


That would require a little refactoring to make it work.  However, the change is simpler if each executor has its own atexit handler:


class ThreadPoolExecutor(_base.Executor):

    def __init__(self, ...):
        ...
        threading._register_atexit(self._atexit())

    def _atexit(self):
        global _shutdown
        _shutdown = True
        self.shutdown()


The value of either approach is that you can then subclass ThreadPoolExecutor to get what you want:


class MyExecutor(ThreadPoolExecutor):
    def shutdown(self, *args, **kwargs):
        stop_my_tasks()
        super().shutdown(*args, **kwwargs)


----

One thing I thought about was supporting a per-task finalizer instead, since that aligns more closely with the way ThreadPoolExecutor works.  It would only apply  So something like one of the following:

* ThreadPoolExecutor(finalize_task=<callable>)
* ThreadPoolExecutor.submit(finalize=<callable)

----

Other things that could be helpful:

* always cancel all the pending tasks during shutdown (and maybe let users opt out)
* use a timeout during shutdown

----

FWIW, adding support for some sort of sub-atexit handler isn't so appealing.  I'm talking about something like one of the following:

* ThreadPoolExecutor(onshutdown=<callable>)
* ThreadPoolExecutor.register_atexit(<callable>)
* (classmethod) ThreadPoolExecutor.register_atexit(<callable>)
* concurrent.futures.register_atexit(<callable>)

(It would probably make sense to pass the list of currently running tasks to the callable.)