classification
Title: Annotating a function as returning an (async) context manager?
Type: Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jelle Zijlstra, barry, levkivskyi, ncoghlan, njs, yselivanov
Priority: normal Keywords:

Created on 2017-05-13 02:48 by njs, last changed 2017-05-21 04:58 by Jelle Zijlstra.

Messages (3)
msg293592 - (view) Author: Nathaniel Smith (njs) * Date: 2017-05-13 02:48
sphinxcontrib-trio [1] does a few things; one of them is to enhance sphinx's autodoc support by trying to sniff out the types of functions so that it can automatically determine that something is, say, a generator, or an async classmethod.

This runs into problems when it comes to context managers, for two reasons. One reason is pretty obvious: the sniffing logic would like to be able to tell by looking at a function that it should be used as a context manager, so that it can document it as such, but there's no reliable way to do this. The other reason is more subtle: the sniffing code has to walk the .__wrapped__ chain in order to detect things like async classmethods, but when it walks the chain for a @contextmanager function, it finds the underlying generator, and ends up thinking that this function should be documented as returning an iterator, which is just wrong. If we can detect context managers, we can know to ignore any 

Obviously this is always going to be a heuristic process; there's no 100% reliable way to tell what arbitrary wrappers are doing. But the heuristic works pretty well... it's just that right now the method of detecting context manager functions is pretty disgusting:

https://github.com/python-trio/sphinxcontrib-trio/blob/2d9e65187dc7a08863b68a78bdee4fb051f0b99e/sphinxcontrib_trio/__init__.py#L80-L90
https://github.com/python-trio/sphinxcontrib-trio/blob/2d9e65187dc7a08863b68a78bdee4fb051f0b99e/sphinxcontrib_trio/__init__.py#L241-L246

So it would be really nice if contextlib.contextmanager somehow marked its functions because:

- then I could (eventually) use that marker instead of making assumptions about contextmanager's internals and messing with code objects

- then we could (immediately) point to how the standard library does it as a standard for other projects to follow, instead of using this weird sphinxcontrib-trio-specific __returns_contextmanager__ thing that I just made up.

I don't really care what the marker is, though I suppose __returns_contextmanager__ = True is as good as anything and has the advantage of a massive installed base (2 projects).

[1] http://sphinxcontrib-trio.readthedocs.io/
msg293596 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-05-13 05:51
Given that we have contextlib.AbstractContextManager now, perhaps it should just set "f.__annotations__['return'] = AbstractContextManager"? (assuming no other return annotation is already set)
msg293964 - (view) Author: Ivan Levkivskyi (levkivskyi) * Date: 2017-05-19 18:53
Or you can use typing.ContextManager[ret_type] if you like generics
(typing.AsyncContextManager will be also added soon).

Also this recent discussion seems relevant https://github.com/python/peps/pull/242 and the corresponding thread on python-dev: https://www.mail-archive.com/python-dev@python.org/msg95595.html
History
Date User Action Args
2017-05-21 04:58:01Jelle Zijlstrasetnosy: + Jelle Zijlstra
2017-05-19 19:08:07terry.reedysettitle: A standard convention for annotating a function as returning an (async) context manager? -> Annotating a function as returning an (async) context manager?
2017-05-19 18:53:47levkivskyisetnosy: + levkivskyi
messages: + msg293964
2017-05-13 05:51:03ncoghlansetmessages: + msg293596
2017-05-13 03:14:18barrysetnosy: + barry
2017-05-13 02:48:47njscreate