There is actually a second thread on python-ideas on a very similar topic, see

    http://mail.python.org/pipermail/python-ideas/2011-December/013021.html

The two main advantages of the proposed CleanupManager over try/finally blocks are

1. You can add a clean-up function conditionally.  In a try/finally block, you would need a flag, which would scatter the single idea more across the code.  Example:

    with CleanupManager() als mngr:
        if f is None:
            f = open("some_file")
            mngr.register(f.close)
        # do something with f (possibly many lines of code)

seems much clearer to me than

    f_needs_closing = False
    if f is None:
        f = open("some_file")
        f_needs_closing = True
    try:
        # do something with f (possibly many lines of code)
    finally:
        if f_needs_closing:
            f.close()

The first version is also much more in the spirit of context managers.  You state at the beginning "we open the file, and guarantee that it will be closed", and we know right from the start that we don't have to bother with it again.

2. CleanupManager could replace several nested try/finally blocks, which again might lead to more readable code.

On the other hand, people who never saw ContextManager before will have to look it up, which will impair readability for those people.

Adding this as a cookbook recipe first seems like a good idea.