Hello,

Last several days I'm trying to implement an async "opener" object that can be used as Coroutine as well as an AsyncContextManager (eg. work with `await obj.open()` and `async with obj.open()`). I've researched several implementations from various python packages such as:
1. aiofiles: https://github.com/Tinche/aiofiles/blob/master/src/aiofiles/base.py#L28
2. aiohttp: https://github.com/aio-libs/aiohttp/blob/master/aiohttp/client.py#L1082

Unlike these libs though, I want my implementation to return a custom object that is a wrapper around the object returned from the underlying module I'm hiding. 

Example:
I want to implement a DataFeeder interface that has a single method `open()`. Sub-classes of this interface will support, for example, opening an file using aiofiles package. So,
AsyncFileDataFeeder.open() will call `aiofiles.open()`, but instead of returning "file-handle" from aiofiles, I want to return a custom Feed class that implements some more methods for reading -- for example:

async with async_data_feeder.open() as feed:
  async for chunk in feed.iter_chunked():
    ...

To support that I'm returning an instance of the following class from DataFeeder.open():

class ContextOpener(
    Coroutine[Any, Any, Feed],
    AbstractAsyncContextManager[Feed],
):
    __slots__ = ("_wrapped_coro", "_feed_cls", "_feed")

    def __init__(self, opener_coro: Coroutine, feed_cls: Type[Feed]):
        self._wrapped_coro = opener_coro
        self._feed_cls = feed_cls

        self._feed: Any = None

    def __await__(self) -> Generator[Any, Any, Feed]:
        print("in await", locals())
        handle = yield from self._wrapped_coro.__await__()
        return self._feed_cls(handle)

    def send(self, value: Any) -> Any:
        print("in send", locals())
        return self._wrapped_coro.send(value)

    def throw(self, *args, **kwargs) -> Any:
        print("in throw", locals())
        return self._wrapped_coro.throw(*args, **kwargs)

    def close(self) -> None:
        print("in close", locals())
        self._wrapped_coro.close()

    async def __aenter__(self) -> feeds.Feed:
        handle = await self._wrapped_coro
        self._feed = self._feed_cls(handle)
        return self._feed

    async def __aexit__(
        self,
        exc_type: Optional[Type[BaseException]],
        exc: Optional[BaseException],
        tb: Optional[TracebackType],
    ) -> None:
        await self._feed.close()
        self._feed = None


This code actually works! But I've noticed that when calling `await DataFeeder.open()` the event loop never calls my `send()` method.

if __name__ == "__main__":
    async def open_test():
        await asyncio.sleep(1)
        return 1

    async def main():
        c = ContextOpener(open_test(), feeds.AsyncFileFeed)
        ret = await c
        print("Finish:", ret, ret._handle)

The output:
in await {'self': <__main__.ContextOpener object at 0x11099cd10>}
Finish: <feeds.AsyncFileFeed object at 0x1109a9a80> 1

From then on a great thinking and reading on the Internet happened, trying to explain to myself how exactly coroutines are working. I suspect that the ContextOpener.__await__ is returning a generator instance and from then on, outer coroutines (eg. main in this case) are calling send()/throw()/close() on THAT generator, not on the ContextOpener "coroutine".
The only way to make Python call ContextOpener send() method (and friends) is when ContextOpener is the outermost coroutine that is communicating directly with the event loop:

ret = asyncio.run(ContextOpener(open_test(), feeds.AsyncFileFeed))
print("Finish:", ret)

Output:
in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None}
in send {'self': <__main__.ContextOpener object at 0x10dcf47c0>, 'value': None}
Finish: 1

However, now I see that I have an error in my implementation that was hidden before: my send() method implementation is not complete because StopIteration case is not handled and returns 1, instead of Feed object.

Since __await__() should return iterator (by PEP492) I can't figure out a way to implement what I want unless making my coroutine class an iterator itself (actually generator) by returning `self` from __await__ and add __iter__ and __next__ methods:

    def __await__(self):
        return self

    def __iter__(self):
        return self

    def __next__(self):
        return self.send(None)

Is this the proper way to make a Coroutine out of a collections.abc.Coroutine? Why is then the documentation not explicitly saying that a Coroutine should inherit from collections.abc.Generator?

I see this as very common misconception since every such "ContextManager" similar to ContextOpener from 3rd party packages (like the aforementioned two, aiofiles and aiohttp, but there are others as well) is subclassing collections.abc.Coroutine and implements send(), throw() and close() methods that are not actually being called. I suspect, the authors of these libraries haven't noticed that because the returned value from the __await__() and send() methods is the same in their case.

You don't need send()/throw()/close() methods.
aiohttp had them to work with Python 3.5

P.S. Please don't use the bug tracker as Q&A site.

Hello Andrew, 

I'm sorry for using the bug tracker, but I wasn't sure whether posting on StackOverflow (is this the appropriate Q&A site?) will attract the attention of the right people.

Although I see you marked this as Closed, I'll be very pleased if you'd elaborate on why I don't need send/throw/close methods since they are "mandatory" from collections.abc.Coroutine. Are you suggesting that I need to just inherit from Awaitable? 

As a matter of fact, I'm writing in the bug tracker because I think that there is a potential for a change -- maybe just in the documentation. I struggle to find an example on how to implement a Coroutine class, and I believe this will be valuable to others, too. While reading the documentation of collections.abc.Coroutine one have an impression that he/she would have to return an Iterator from __await__() and then outer coroutines will call send/throw/close methods of the derived collections.abc.Coroutine class, but this is not the case. One have to dive deep into Python internals to grasp why Iterator should be returned and how outer coroutines interact with it.

Moreover, there are several cross-referencing PEPs that explain yield, yield from, await but none of them has an example of how to construct a Coroutine from a class. Explanation and examples are always concerned with `yield` and its suspension property is presented as implementation detail.

> Are you suggesting that I need to just inherit from Awaitable? 

Yes. Awaitable is a very base protocol, Coroutine is an implementation.

`__await__` returns a generator that is used by Python internals as if `yield from awaitable.__await__()` was called.  asyncio never sends data back to the generator but other async frameworks can do it.

`send()`/`close()`/`throw()` coroutine methods are never used now IFAIK.  The current implementation calls these methods at the C level without using Python-exposed names. Performance matters.
These methods can be still used in very rare cases, e.g. when explicit Python `yield from custom_coro.__await__()` is used.