Author kristjan.jonsson
Recipients barry, eric.snow, kristjan.jonsson, ncoghlan, r.david.murray
Date 2013-08-14.20:08:25
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1376510906.31.0.942802942615.issue18677@psf.upfronthosting.co.za>
In-reply-to
Content
"locally visible" is, I think a very misleading term.  How is

with ignore_error, acquire_resource as r:
   doo_stuff_with_resource(r) #can be silently skipped

any more locally visible than
with acquire_resource_ignore_error as r:
    doo_stuff_with resource(r) # can be silently skipped.

? does the "nested with" syntax immediatelly tell you "hey, the body can be silently skipped"?

Requiring that some context manager patterns must be done with a special syntax is odd.  What is more, it prohibits us to abstract away context managers.  For instance, you can write a function like this

def execute_with_context(ctxt, fn, args):
    with ctxt:
        return fn(*args)

but if your context manager is of the kind mentioned, i.e. requiring the double syntax, you are screwed.

Basically, what I'm proposing (and what the patch provides) is that you can write this code:
@contextmanager
def nestedc(ca, cb):
    with ca as a, cb as b:
        yield a, b

and have it work for _all_ pair of ca, cb.  This then, allows context managers to be used like abstract entities, like other objects in the language.  It is _not_ about flow control, but about completeness.

A similar pattern for functions is already possible:
def nestedf(fa, fb):
    def helper(v):
        return fa(fb(v))
    return helper

And so, we could write:
execute_with_context(nestedc(ca, cb), nestedf(fa, fb), ('foo',))

Current python does not allow this for arbitrary pairs ca, cb.  My version does.  This is what I'm advocating.  That programmers are given the tool to combine context managers if they want.
 

As for "contextlib.nested()".
I'm not necessarily advocation its resuciation in the standardlib, but adding that to the patch here to demonstrate how it now _works_.

Here is a simpler version of contextlib.nested:

@contextmanager
def nested_empty():
    yield []

@contextmanager
def nested_append(prev, next):
    with prev as a, next as b:
        a.append(b)
        yield a

def nested(*managers):
    total = nested_empty()
    for mgr in managers:
        total = nested_append(total, mgr)
    return total

Pretty nice, no?

Now we come to the argument with nested(open(a), open(b)).
I see your point, but I think that the problem is not due to nested, but to open.  Deprecating nested, even as a programming pattern demonstration is throwing out the baby with the bathwater.

I´ve coined the term "hybrid context manager" (at least I think I have)to mean resources that are their own context managers.  They're hybrid because they are acquired explicitly, but can be released via a context manager.  The context manager is a bolt on, an afterthought.  Instead of adding __exit__() to files, and allowing
with open(fn) as f: pass
We should have encouraged the use of proper context managers:
with opened(fn) as f: pass
or 
with closing(f):  pass

Now, we unfortunately have files being context managers and widely see the pattern
with open(fn) as f, open(fn2) as f2:
    pass

But how is this bug here:
with nested(open(fn), open(fn2)) as f, f2: pass

any more devuiys than
f, f2 = open(fn), open(fn2)
with f, f2: pass
?

The problem is that files aren't "real" context managers but "hybrids" and this is what we should warn people about.  The fact that we do have those hybrids in our code base should not be cause to remove tools that are designed to work with "proper" context managers.

The decision to remove "nested" on these grounds sets the precedence that we cannot have any functions that operate on context managers.  In fact, what this is really is saying is this:

"context managers should only be used with the "with" statement and only instantiated in-line.
Anything else may introduce sublte bugs because some context managers are in fact not context managers, but the resource that they manage.
"

In my opinion, it would have been better to deprecate the use of files as context managers, and instead urge people to use proper context managers for the:  (the proposed) contextlib.opened and (the existing) contextlib.closing)

K
History
Date User Action Args
2013-08-14 20:08:26kristjan.jonssonsetrecipients: + kristjan.jonsson, barry, ncoghlan, r.david.murray, eric.snow
2013-08-14 20:08:26kristjan.jonssonsetmessageid: <1376510906.31.0.942802942615.issue18677@psf.upfronthosting.co.za>
2013-08-14 20:08:26kristjan.jonssonlinkissue18677 messages
2013-08-14 20:08:25kristjan.jonssoncreate