Title: atexit module: allow getting/setting list of handlers directly
Type: behavior Stage:
Components: Library (Lib) Versions:
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Xtrem532, chrahunt, erik.bray, iritkatriel, mark.dickinson
Priority: normal Keywords:

Created on 2017-11-20 09:54 by erik.bray, last changed 2021-08-22 12:37 by mark.dickinson.

Messages (5)
msg306537 - (view) Author: Erik Bray (erik.bray) * (Python triager) Date: 2017-11-20 09:54
In Python 2 it was possible to directly manipulate the list of registered atexit handlers through atexit._exithandlers.  Obviously I'm not complaining that this went away, as it was an underscored attribute.  But this possibility was still useful in the context of testing.

For example, we have a test suite that runs many test cases in subprocesses run with multiprocessing.Process.  Since these call os._exit() any atexit handlers registered by the code under test are not run.  It's useful, however, to test that either

a) Expected atexit handlers ran correctly or
b) No unexpected atexit handlers were registered

To this end we would save and clear atexit._exithandlers, call atexit._run_exitfuncs(), then restore the original atexit._exithandlers.

This is not possible on Python 3 since that all lives in the C module state for sub-interpreter support.  For the time being it was necessary to work around this with a Cython module, but coding around internal extension module structures is hardly ideal:

I think it would be useful--for testing purposes *only*--to add a _get_exitfuncs() function that returns a tuple of the registered handlers, and likewise a _set_exitfuncs(handlers) with sets the registered handlers from an iterable (the latter being little more than a shortcut for `atexit._clear(); for h in handlers: atexit.register(*h)`).

I would propose that these be undocumented internal functions to emphasize that they are not how the module should be used in normal circumstances.  At the same time it might be worth addressing  I can provide a patch if this idea is acceptable.
msg347292 - (view) Author: Christopher Hunt (chrahunt) * Date: 2019-07-04 18:42
Updated link to workaround referenced in the original issue:
msg394653 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-05-28 12:06
You say "for testing purposes only", but I'd add "for debugging" to the list of use-cases, having just spent a few hours trying to understand why I was getting a particular PySide2 "Internal C++ object already deleted." exception under Python 3.6 but not under Python 3.7 and later. [*]

Even just making the list of exit handlers introspectable (so just _get_exitfuncs()) would have been really useful in diagnosing the issue faster.

[*] The answer turned out to be that under Python 3.6 the PySide2 exit handler was being run before the concurrent.futures thread-cleanup exit handler, while on Python 3.7, for obscure reasons, the exit handlers were being registered in the reverse order. To discover this, I ended up resorting to the elegant hack (if that's not a contradiction in terms) described in
msg396593 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-27 18:46
See also issue17186.
msg400071 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-08-22 12:37
Thanks, Irit. Should we close this issue as a duplicate? It's not identical (being able to modify the list of handlers versus just being able to inspect it), but it's pretty close.
Date User Action Args
2021-08-22 12:37:01mark.dickinsonsetmessages: + msg400071
2021-06-27 18:46:53iritkatrielsetnosy: + iritkatriel
messages: + msg396593
2021-05-28 12:06:47mark.dickinsonsetnosy: + mark.dickinson
messages: + msg394653
2020-11-20 14:42:35Xtrem532setnosy: + Xtrem532
2019-07-04 18:42:05chrahuntsetnosy: + chrahunt
messages: + msg347292
2017-11-20 09:54:41erik.braycreate