Message408321
Hi folks!
Let me first present an example that motivated this issue. Imagine a script that builds Docker images and later starts them as Docker containers. To avoid having to stop the containers "manually" (in code and potentially forgot) I had an idea to register each container when started and stop each using atexit module. I chose to encapsulate this behavior inside a class. A much simplified example looks like this:
import atexit
class Program:
# keep a class level list of started containers
running_containers = []
@atexit.register
@classmethod
def clean_up(cls, *args, **kwargs):
for container in cls.running_containers:
print(f'stopping {container}')
def start_container(self, container):
print(f'starting {container}')
self.__class__.running_containers.append(container)
prog = Program()
a.start_container('container_A')
a.start_container('container_B')
And I'd expect this to produce:
starting container_A
starting container_B
stopping container_A
stopping container_B
To me, this reads rather nicely: the Program.clean_up method can be called by the user, but if he forgets it will be handled for him using atexit. However, this code does not work. :) I've spent some time debugging and what follows are my observations:
1) If the order of decorators is @atexit.register and then @classmethod then the code throws 'TypeError: the first argument must be callable'. I believe it is because classmethod and staticmethod are descriptors without the __call__ method implemented. atexit.register does not check this and instead of func.__func__ (which resolves to Program.clean_up) gets func (a classmethod) which is not callable (https://github.com/python/cpython/blob/main/Modules/atexitmodule.c#L147).
2) If the order of decorators is swapped (@classmethod and @atexit.register) then the code throws "Error in atexit._run_exitfuncs:\nTypeError: clean_up() missing 1 required positional argument: 'cls'". From my limited understanding of CPython and atexitmodule.c I think what happens is that the @atexit.register returns (https://github.com/python/cpython/blob/main/Modules/atexitmodule.c#L180) the func without the args and kwargs (since this issue https://bugs.python.org/issue1597824).
3) However, if I step away from decorating using @atexit.register and instead use
[...]
atexit.register(Program.clean_up) # <-- register post definition
prog = Program()
a.start_container('container_A')
a.start_container('container_B')
then the code works as expected and outputs:
starting container_A
starting container_B
stopping container_A
stopping container_B
To summarize, I don't like 3) as it puts the responsibility in a bit awkward place (good enough if I'm the only user, but I wonder about the more general library-like cases). My decorating skills are a bit dull now and it's my first time seriously looking into CPython internals - I've tried to encapsulate atexit.register in my custom decorator, to check whether that could be a workaround but overall was unsuccessful. In short, I'd say that in both 1) and 2) the cls arg is lost when atexit calls the function. I've tried to dig it up from the func passed to atexit.register
def my_atexit_decorator(func, *args, **kwargs):
cls = # some magic with under attrs and methods
register.atexit(func, cls=cls, *args, **kwargs)
[...]
, but failed (it also felt like a fragile approach).
I was not able to understand why @atexit.register does not work when the function's signature is not empty. Also, if fixable I'm happy to actually put the effort into fixing it myself (looks like a nice first CPython PR), but I'd like to have someone else's opinion before I start marching in the wrong direction. Also, let me know if you'd like more details or code/tests I've produced while debugging this.
Cheers! |
|
Date |
User |
Action |
Args |
2021-12-11 17:27:01 | quapka | set | recipients:
+ quapka |
2021-12-11 17:27:01 | quapka | set | messageid: <1639243621.71.0.144294694671.issue46051@roundup.psfhosted.org> |
2021-12-11 17:27:01 | quapka | link | issue46051 messages |
2021-12-11 17:27:01 | quapka | create | |
|