My earlier descriptions here aren't really adequate - as soon as I started putting contextlib2 together, this CleanupManager idea quickly morphed into ContextStack [1], which is a far more powerful tool for manipulating context managers in a way that doesn't necessarily correspond with lexical scoping in the source code. Using the "collection of files" example:

    with ContextStack() as stack:
        files = [stack.enter_context(open(fname)) for fname in filenames]
        # All opened files will automatically be closed at the end of
        # the with statement, even if attempts to open files later
        # in the list throw an exception

Or the optional resource use case:

    with ContextStack() as stack:
        if resource is None:
            resource = stack.enter_context(make_default_resource())
        # If we created it, the resource will be cleaned up
        # Otherwise, it will be left alone

The "cleanup only if the operation fails" use case (coming in v0.3 [2])

    def open_files(*filenames):        
        """Returns an (opened_files, context_stack) 2-tuple

        The context stack will automatically close all of the opened files.
        If any file fails to open, all previously opened file handles will be released immediately.
        """
        with ContextStack() as stack:
            files = [stack.enter_context(open(fname)) for fname in filenames]
            return files, stack.preserve()

The "don't repeat your __exit__ code in __enter__" use case:

    def __enter__(self):
        resource = self._acquire_resource()
        with ContextStack() as stack:
            stack.register_exit(self.__exit__)
            self._check_resource_validity()
            stack.preserve() # All good!
        return resource

(In 0.3, register_exit() will probably check for __exit__ attributes automatically, so it will accept both callables and context managers)

[1] http://contextlib2.readthedocs.org/en/latest/index.html#contextlib2.ContextStack
[2] https://bitbucket.org/ncoghlan/contextlib2/issue/1/add-a-preserve-method-to-relinquish