This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author quapka
Recipients quapka
Date 2021-12-11.17:27:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1639243621.71.0.144294694671.issue46051@roundup.psfhosted.org>
In-reply-to
Content
Hi folks!

Let me first present an example that motivated this issue. Imagine a script that builds Docker images and later starts them as Docker containers. To avoid having to stop the containers "manually" (in code and potentially forgot) I had an idea to register each container when started and stop each using atexit module. I chose to encapsulate this behavior inside a class. A much simplified example looks like this:

    import atexit

    class Program:
        # keep a class level list of started containers
        running_containers = []

        @atexit.register
        @classmethod
        def clean_up(cls, *args, **kwargs):
            for container in cls.running_containers:
                print(f'stopping {container}')

        def start_container(self, container):
            print(f'starting {container}')
            self.__class__.running_containers.append(container)

    prog = Program()
    a.start_container('container_A')
    a.start_container('container_B')

And I'd expect this to produce:

    starting container_A
    starting container_B
    stopping container_A
    stopping container_B

To me, this reads rather nicely: the Program.clean_up method can be called by the user, but if he forgets it will be handled for him using atexit. However, this code does not work. :) I've spent some time debugging and what follows are my observations:

1) If the order of decorators is @atexit.register and then @classmethod then the code throws 'TypeError: the first argument must be callable'. I believe it is because classmethod and staticmethod are descriptors without the __call__ method implemented. atexit.register does not check this and instead of func.__func__ (which resolves to Program.clean_up) gets func (a classmethod) which is not callable (https://github.com/python/cpython/blob/main/Modules/atexitmodule.c#L147).

2) If the order of decorators is swapped (@classmethod and @atexit.register) then the code throws "Error in atexit._run_exitfuncs:\nTypeError: clean_up() missing 1 required positional argument: 'cls'". From my limited understanding of CPython and atexitmodule.c I think what happens is that the @atexit.register returns (https://github.com/python/cpython/blob/main/Modules/atexitmodule.c#L180) the func without the args and kwargs (since this issue https://bugs.python.org/issue1597824).

3) However, if I step away from decorating using @atexit.register and instead use

    [...]
    atexit.register(Program.clean_up) # <-- register post definition
    prog = Program()
    a.start_container('container_A')
    a.start_container('container_B')

then the code works as expected and outputs:

    starting container_A
    starting container_B
    stopping container_A
    stopping container_B


To summarize, I don't like 3) as it puts the responsibility in a bit awkward place (good enough if I'm the only user, but I wonder about the more general library-like cases). My decorating skills are a bit dull now and it's my first time seriously looking into CPython internals - I've tried to encapsulate atexit.register in my custom decorator, to check whether that could be a workaround but overall was unsuccessful. In short, I'd say that in both 1) and 2) the cls arg is lost when atexit calls the function. I've tried to dig it up from the func passed to atexit.register

    def my_atexit_decorator(func, *args, **kwargs):
        cls = # some magic with under attrs and methods
        register.atexit(func, cls=cls, *args, **kwargs)
        [...]

, but failed (it also felt like a fragile approach).

I was not able to understand why @atexit.register does not work when the function's signature is not empty. Also, if fixable I'm happy to actually put the effort into fixing it myself (looks like a nice first CPython PR), but I'd like to have someone else's opinion before I start marching in the wrong direction. Also, let me know if you'd like more details or code/tests I've produced while debugging this.

Cheers!
History
Date User Action Args
2021-12-11 17:27:01quapkasetrecipients: + quapka
2021-12-11 17:27:01quapkasetmessageid: <1639243621.71.0.144294694671.issue46051@roundup.psfhosted.org>
2021-12-11 17:27:01quapkalinkissue46051 messages
2021-12-11 17:27:01quapkacreate