classification
Title: Add `pop` function to remove context manager from (Async)ExitStack
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: andreash, ncoghlan, serhiy.storchaka, yselivanov
Priority: normal Keywords:

Created on 2021-09-13 14:40 by andreash, last changed 2021-09-13 19:42 by andreash.

Messages (3)
msg401703 - (view) Author: Andreas H. (andreash) Date: 2021-09-13 14:40
Currently it is not possible to remove context managers from an ExitStack (or AsyncExitStack). 


Workarounds are difficult and generally do accesses implementation details of (Async)ExitStack.
See e.g. https://stackoverflow.com/a/37607405. It could be done as follows:


class AsyncExitStackWithPop(contextlib.AsyncExitStack):
    """Same as AsyncExitStack but with pop, i.e. removal functionality"""
    async def pop(self, cm):
        callbacks = self._exit_callbacks
        self._exit_callbacks = collections.deque()
        found = None
        while callbacks:
            cb = callbacks.popleft()
            if cb[1].__self__ == cm:
                found = cb
            else:
                self._exit_callbacks.append(cb)
        if not found:
            raise KeyError("context manager not found")
        if found[0]:
            return found[1](None,None,None)
        else:
            return await found[1](None, None, None)

The alternative is re-implementation of ExitStack with pop functionality, but that is also very difficult to get right (especially with exceptions). Which is probably the reason why there is ExitStack in the library at all.


So I propose to augment (Async)ExitStack with a `pop` method like above or similar to the above.


Use-Cases:

An example is a component that manages several connections to network services. 
During run-time the network services might need to change (i.e. some be disconnected and some be connected according to business logic), or handle re-connection events (ie. graceful response to network errors).
It is not too hard to imagine more use cases.
Essentially every case where dynamic resource management is needed and where single resources are managable with python context managers.
msg401708 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-13 16:22
If you can remove context managers in arbitrary order it is no longer a stack.

I think that it is just a wrong tool for your problem. Just maintain a collection of opened connections. Set or dict can be better that list or deque.
msg401722 - (view) Author: Andreas H. (andreash) Date: 2021-09-13 19:42
I see your point. But even with `pop` or `remove` it is still a stack or stack-like. In the normal case the context managers are still released in reverse order as they were added. Order cannot be changed arbitrarily.

There is just the additional function of removing a single context manager prematurely(e.g. for graceful error recovery and such). 

I would perhaps say that a stack is the "wrong" solution to the problem of "programmatically combining context managers" [this is from the official documentaion] in the first place. I write wrong in quotes because it is of course not really wrong, as one wants the reverse exit order. But to adequately address the dynamic case one needs in my opinion the ability to prematurely remove context managers. Otherwise the use is limited.

Reimplemeting the desired functionality with dicts or lists does not seem appealing to me as the code will be 90% the same to ExitStack. It will then also make ExitStack obsolete. So why not integrate it there?

The unsymmetry of being able to add context managers but not being able to remove them also seems odd to me.
History
Date User Action Args
2021-09-13 19:42:26andreashsetmessages: + msg401722
2021-09-13 16:22:04serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg401708
2021-09-13 14:40:02andreashcreate