diff -r 04cc39ac03b5 Doc/c-api/typeobj.rst --- a/Doc/c-api/typeobj.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/c-api/typeobj.rst Thu May 21 02:06:07 2015 -0400 @@ -220,9 +220,16 @@ the subtype's :c:member:`~PyTypeObject.tp_setattr` and :c:member:`~PyTypeObject.tp_setattro` are both *NULL*. -.. c:member:: void* PyTypeObject.tp_reserved +.. c:member:: void* PyTypeObject.tp_as_async - Reserved slot, formerly known as tp_compare. + Pointer to an additional structure that contains fields relevant only to + objects which implement :term:`awaitable` and :term:`asynchronous iterator` + protocols at the C-level. See :ref:`async-structs` for details. + + .. versionadded:: 3.5 + + .. note:: + Formerly known as tp_compare and tp_reserved. .. c:member:: reprfunc PyTypeObject.tp_repr @@ -1332,3 +1339,57 @@ :c:func:`PyBuffer_Release` is the interface for the consumer that wraps this function. + + +.. _async-structs: + + +Async Object Structures +======================= + +.. sectionauthor:: Yury Selivanov + + +.. c:type:: PyAsyncMethods + + This structure holds pointers to the functions required to implement + :term:`awaitable` and :term:`asynchronous iterator` objects. + + Here is the structure definition:: + + typedef struct { + getawaitablefunc am_await; + getaiterfunc am_aiter; + aiternextfunc am_anext; + } PyAsyncMethods; + +.. c:member:: getawaitablefunc PyAsyncMethods.am_await + + The signature of this function is:: + + PyObject *am_await(PyObject *self) + + The returned object must be an iterator, i.e. :c:func:`PyIter_Check` must + return ``1`` for it. + + This slot may be set to *NULL* if an object is not an :term:`awaitable`. + +.. c:member:: getaiterfunc PyAsyncMethods.am_aiter + + The signature of this function is:: + + PyObject *am_aiter(PyObject *self) + + Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + + This slot may be set to *NULL* if an object does not implement + asynchronous iteration protocol. + +.. c:member:: aiternextfunc PyAsyncMethods.am_anext + + The signature of this function is:: + + PyObject *am_anext(PyObject *self) + + Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + This slot may be set to *NULL*. diff -r 04cc39ac03b5 Doc/extending/newtypes.rst --- a/Doc/extending/newtypes.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/extending/newtypes.rst Thu May 21 02:06:07 2015 -0400 @@ -80,7 +80,7 @@ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */ - 0, /* tp_reserved */ + 0, /* tp_as_async */ 0, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ diff -r 04cc39ac03b5 Doc/glossary.rst --- a/Doc/glossary.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/glossary.rst Thu May 21 02:06:07 2015 -0400 @@ -69,11 +69,42 @@ :ref:`the difference between arguments and parameters `, and :pep:`362`. + asynchronous context manager + An object which controls the environment seen in a + :keyword:`async with` statement by defining :meth:`__aenter__` and + :meth:`__aexit__` methods. Introduced by :pep:`492`. + + .. versionadded:: 3.5 + + asynchronous iterable + An object, that can be used in an :keyword:`async for` statement. + Must return an :term:`awaitable` from its :meth:`__aiter__` method, + which should in turn be resolved in an :term:`asynchronous iterator` + object. Introduced by :pep:`492`. + + .. versionadded:: 3.5 + + asynchronous iterator + An object that implements :meth:`__aiter__` and :meth:`__anext__` + methods, that must return :term:`awaitable` objects. + :keyword:`async for` resolves awaitable returned from asynchronous + iterator's :meth:`__anext__` method until it raises + :exc:`StopAsyncIteration` exception. Introduced by :pep:`492`. + + .. versionadded:: 3.5 + attribute A value associated with an object which is referenced by name using dotted expressions. For example, if an object *o* has an attribute *a* it would be referenced as *o.a*. + awaitable + An object that can be used in :keyword:`await` expression. Can be + a :term:`coroutine` or an object with an :meth:`__await__` method. + See also :pep:`492`. + + .. versionadded:: 3.5 + BDFL Benevolent Dictator For Life, a.k.a. `Guido van Rossum `_, Python's creator. @@ -146,6 +177,23 @@ statement by defining :meth:`__enter__` and :meth:`__exit__` methods. See :pep:`343`. + coroutine function + A function which returns a :term:`coroutine` object. It is defined + with an :keyword:`async def` keyword, and may contain :keyword:`await`, + :keyword:`async for`, and :keyword:`async with` keywords. Introduced + by :pep:`492`. + + .. versionadded:: 3.5 + + coroutine + Coroutines is a more generalized form of subroutines. Subroutines are + entered at one point and exited at another point. Coroutines, can be + entered, exited, and resumed at many different points. See + :keyword:`await` expressions, and :keyword:`async for` and + :keyword:`async with` statements. See also :pep:`492`. + + .. versionadded:: 3.5 + CPython The canonical implementation of the Python programming language, as distributed on `python.org `_. The term "CPython" diff -r 04cc39ac03b5 Doc/library/collections.abc.rst --- a/Doc/library/collections.abc.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/library/collections.abc.rst Thu May 21 02:06:07 2015 -0400 @@ -33,9 +33,9 @@ .. tabularcolumns:: |l|L|L|L| -========================= ===================== ====================== ==================================================== +========================== ====================== ======================= ==================================================== ABC Inherits from Abstract Methods Mixin Methods -========================= ===================== ====================== ==================================================== +========================== ====================== ======================= ==================================================== :class:`Container` ``__contains__`` :class:`Hashable` ``__hash__`` :class:`Iterable` ``__iter__`` @@ -81,7 +81,11 @@ :class:`KeysView` :class:`MappingView`, ``__contains__``, :class:`Set` ``__iter__`` :class:`ValuesView` :class:`MappingView` ``__contains__``, ``__iter__`` -========================= ===================== ====================== ==================================================== +:class:`Awaitable` ``__await__`` +:class:`Coroutine` ``send``, ``throw`` ``close`` +:class:`AsyncIterable` ``__aiter__`` +:class:`AsyncIterator` :class:`AsyncIterable` ``__anext__`` ``__aiter__`` +========================== ====================== ======================= ==================================================== .. class:: Container @@ -134,6 +138,41 @@ ABCs for mapping, items, keys, and values :term:`views `. +.. class:: Awaitable + + ABC for classes that provide ``__await__`` method. Instances + of such classes can be used in ``await`` expression. + + :term:`coroutine` objects and instances of + :class:`~collections.abc.Coroutine` are too instances of this ABC. + + .. versionadded:: 3.5 + +.. class:: Coroutine + + ABC for coroutine compatible classes that implement a subset of + generator methods defined in :pep:`342`, namely: + :meth:`~generator.send`, :meth:`~generator.throw` and + :meth:`~generator.close` methods. All :class:`Coroutine` instances + are also instances of :class:`Awaitable`. See also the definition + of :term:`coroutine`. + + .. versionadded:: 3.5 + +.. class:: AsyncIterable + + ABC for classes that provide ``__aiter__`` method. See also the + definition of :term:`asynchronous iterable`. + + .. versionadded:: 3.5 + +.. class:: AsyncIterator + + ABC for classes that provide ``__aiter__`` and ``__anext__`` + methods. See also the definition of :term:`asynchronous iterator`. + + .. versionadded:: 3.5 + These ABCs allow us to ask classes or instances if they provide particular functionality, for example:: diff -r 04cc39ac03b5 Doc/library/exceptions.rst --- a/Doc/library/exceptions.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/library/exceptions.rst Thu May 21 02:06:07 2015 -0400 @@ -322,6 +322,14 @@ .. versionchanged:: 3.5 Introduced the RuntimeError transformation. +.. exception:: StopAsyncIteration + + Must be raised by :meth:`__anext__` method of an + :term:`asynchronous iterator` object to stop the iteration. + + .. versionadded:: 3.5 + See also :pep:`492`. + .. exception:: SyntaxError Raised when the parser encounters a syntax error. This may occur in an diff -r 04cc39ac03b5 Doc/library/inspect.rst --- a/Doc/library/inspect.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/library/inspect.rst Thu May 21 02:06:07 2015 -0400 @@ -266,6 +266,41 @@ Return true if the object is a generator. +.. function:: iscoroutinefunction(object) + + Return true if the object is a coroutine function. + + Coroutine functions are defined with "async def" syntax, + or are generators decorated with :func:`types.coroutine` + or :func:`asyncio.coroutine`. + + The function will return false for plain python generator + functions. + + .. versionadded:: 3.5 + + +.. function:: iscoroutine(object) + + Return true if the object is a coroutine. + + Coroutines are results of calls of coroutine functions or + generator functions decorated with :func:`types.coroutine` + or :func:`asyncio.coroutine`. + + The function will return false for plain python generators. + + .. versionadded:: 3.5 + + +.. function:: isawaitable(object) + + Return true if the object can be used in :keyword:`await` + expression. See also :class:`collections.abc.Awaitable`. + + .. versionadded:: 3.5 + + .. function:: istraceback(object) Return true if the object is a traceback. diff -r 04cc39ac03b5 Doc/library/types.rst --- a/Doc/library/types.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/library/types.rst Thu May 21 02:06:07 2015 -0400 @@ -271,3 +271,17 @@ attributes on the class with the same name (see Enum for an example). .. versionadded:: 3.4 + + +Coroutines Utility Functions +---------------------------- + +.. function:: coroutine(gen_func) + + The function transforms a generator function to a :term:`coroutine function`, + so that it returns a :term:`coroutine` object. + + *gen_func* is modified in-place, hence the function can be used as a + decorator. + + .. versionadded:: 3.5 diff -r 04cc39ac03b5 Doc/reference/compound_stmts.rst --- a/Doc/reference/compound_stmts.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/reference/compound_stmts.rst Thu May 21 02:06:07 2015 -0400 @@ -51,6 +51,9 @@ : | `with_stmt` : | `funcdef` : | `classdef` + : | `async_with_stmt` + : | `async_for_stmt` + : | `async_funcdef` suite: `stmt_list` NEWLINE | NEWLINE INDENT `statement`+ DEDENT statement: `stmt_list` NEWLINE | `compound_stmt` stmt_list: `simple_stmt` (";" `simple_stmt`)* [";"] @@ -660,6 +663,112 @@ :pep:`3129` - Class Decorators +Coroutines +========== + +.. _`async def`: + +Coroutine function definition +----------------------------- + +.. productionlist:: + async_funcdef: "async" `funcdef` + +Execution of Python coroutines can be suspended and resumed at many points +(see :term:`coroutine`.) :keyword:`await` expressions, :keyword:`async for` +and :keyword:`async with` can only be used in their bodies. + +Functions defined with ``async def`` syntax are always coroutine functions, +even if they do not contain ``await`` or ``async`` keywords. + +It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in coroutines. + +.. versionadded:: 3.5 + + +.. _`async for`: + +The :keyword:`async for` statement +---------------------------------- + +.. productionlist:: + async_for_stmt: "async" `for_stmt` + +An :term:`asynchronous iterable` is able to call asynchronous code in its +*iter* implementation, and :term:`asynchronous iterator` can call asynchronous +code in its *next* method. + +The ``async for`` statement allows convenient iteration over asynchronous +iterators. + +The following code:: + + async for TARGET in ITER: + BLOCK + else: + BLOCK2 + +Is semantically equivalent to:: + + iter = (ITER) + iter = await type(iter).__aiter__(iter) + running = True + while running: + try: + TARGET = await type(iter).__anext__(iter) + except StopAsyncIteration: + running = False + else: + BLOCK + else: + BLOCK2 + +See also :meth:`__aiter__` and :meth:`__anext__` for details. + +.. versionadded:: 3.5 + + +.. _`async with`: + +The :keyword:`async with` statement +----------------------------------- + +.. productionlist:: + async_with_stmt: "async" `with_stmt` + +An :term:`asynchronous context manager` is a :term:`context manager` that is +able to suspend execution in its *enter* and *exit* methods. + +The following code:: + + async with EXPR as VAR: + BLOCK + +Is semantically equivalent to:: + + mgr = (EXPR) + aexit = type(mgr).__aexit__ + aenter = type(mgr).__aenter__(mgr) + exc = True + + VAR = await aenter + try: + BLOCK + except: + if not await aexit(mgr, *sys.exc_info()): + raise + else: + await aexit(mgr, None, None, None) + +See also :meth:`__aenter__` and :meth:`__aexit__` for details. + +.. versionadded:: 3.5 + +.. seealso:: + + :pep:`492` - Coroutines with async and await syntax + + .. rubric:: Footnotes .. [#] The exception is propagated to the invocation stack unless diff -r 04cc39ac03b5 Doc/reference/datamodel.rst --- a/Doc/reference/datamodel.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/reference/datamodel.rst Thu May 21 02:06:07 2015 -0400 @@ -616,6 +616,16 @@ exception is raised and the iterator will have reached the end of the set of values to be returned. + Coroutine functions + .. index:: + single: coroutine; function + + A function or method which is defined using :keyword:`async def` is called + a :dfn:`coroutine function`. Such a function, when called, returns a + :term:`coroutine` object. It may contain :keyword:`await` expressions, + as well as :keyword:`async with` and :keyword:`async for` statements. See + also :ref:`coroutines` section. + Built-in functions .. index:: object: built-in function @@ -2254,6 +2264,104 @@ object itself in order to be consistently invoked by the interpreter). +.. _coroutines: + +Coroutines +========== + +.. index:: + single: coroutine + + +Awaitable Objects +----------------- + +An *awaitable* object can be one of the following: + +* A :term:`coroutine` object returned from a :term:`coroutine function`. + +* A :term:`generator` decorated with :func:`types.coroutine` + (or :func:`asyncio.coroutine`) decorator. + +* An object that implements an ``__await__`` method. + +.. method:: object.__await__(self) + + Must return an :term:`iterator`. Should be used to implement + :term:`awaitable` objects. For instance, :class:`asyncio.Future` implements + this method to be compatible with the :keyword:`await` expression. + +.. versionadded:: 3.5 + +.. seealso:: :pep:`492` for additional information about awaitable objects. + + +Asynchronous Iterators +---------------------- + +An *asynchronous iterable* is able to call asynchronous code in its +``__aiter__`` implementation, and an *asynchronous iterator* can call +asynchronous code in its ``__anext__`` method. + +Asynchronous iterators can be used in a :keyword:`async for` statement. + +.. method:: object.__aiter__(self) + + Must return an *awaitable* resulting in an *asynchronous iterator* object. + +.. method:: object.__anext__(self) + + Must return an *awaitable* resulting in a next value of the iterator. Should + raise a :exc:`StopAsyncIteration` error when the iteration is over. + +An example of an asynchronous iterable object:: + + class Reader: + async def readline(self): + ... + + async def __aiter__(self): + return self + + async def __anext__(self): + val = await self.readline() + if val == b'': + raise StopAsyncIteration + return val + +.. versionadded:: 3.5 + + +Asynchronous Context Managers +----------------------------- + +An *asynchronous context manager* is a *context manager* that is able to +suspend execution in its ``__aenter__`` and ``__aexit__`` methods. + +Asynchronous context managers can be used in a :keyword:`async with` statement. + +.. method:: object.__aenter__(self) + + This method is semantically similar to the :meth:`__enter__`, with only + difference that it must return an *awaitable*. + +.. method:: object.__aexit__(self, exc_type, exc_value, traceback) + + This method is semantically similar to the :meth:`__exit__`, with only + difference that it must return an *awaitable*. + +An example of an asynchronous context manager class:: + + class AsyncContextManager: + async def __aenter__(self): + await log('entering context') + + async def __aexit__(self, exc_type, exc, tb): + await log('exiting context') + +.. versionadded:: 3.5 + + .. rubric:: Footnotes .. [#] It *is* possible in some cases to change an object's type, under certain diff -r 04cc39ac03b5 Doc/reference/expressions.rst --- a/Doc/reference/expressions.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/reference/expressions.rst Thu May 21 02:06:07 2015 -0400 @@ -811,6 +811,20 @@ if that method was called. +.. _await: + +Await expression +================ + +Suspend the execution of :term:`coroutine` on an :term:`awaitable` object. +Can only be used inside a :term:`coroutine function`. + +.. productionlist:: + await: ["await"] `primary` + +.. versionadded:: 3.5 + + .. _power: The power operator @@ -820,7 +834,7 @@ less tightly than unary operators on its right. The syntax is: .. productionlist:: - power: `primary` ["**" `u_expr`] + power: `await` ["**" `u_expr`] Thus, in an unparenthesized sequence of power and unary operators, the operators are evaluated from right to left (this does not constrain the evaluation order @@ -1362,6 +1376,8 @@ +-----------------------------------------------+-------------------------------------+ | ``**`` | Exponentiation [#]_ | +-----------------------------------------------+-------------------------------------+ +| ``await`` ``x`` | Await expression | ++-----------------------------------------------+-------------------------------------+ | ``x[index]``, ``x[index:index]``, | Subscription, slicing, | | ``x(arguments...)``, ``x.attribute`` | call, attribute reference | +-----------------------------------------------+-------------------------------------+ diff -r 04cc39ac03b5 Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst Wed May 20 22:09:43 2015 -0500 +++ b/Doc/whatsnew/3.5.rst Thu May 21 02:06:07 2015 -0400 @@ -121,6 +121,25 @@ PEP written by Carl Meyer +PEP 492 - Coroutines with async and await syntax +------------------------------------------------ + +The PEP proposes adding dedicated syntax for declaring coroutines, await +expressions, and new asynchronous *for* and *with* statements. + +Example:: + + async def read_data(db): + async with db.transaction(): + data = await db.fetch('SELECT ...') + +PEP written and implemented by Yury Selivanov. + +.. seealso:: + + :pep:`492` -- Coroutines with async and await syntax + + PEP 461 - Adding formatting to bytes and bytearray -------------------------------------------------- @@ -961,4 +980,5 @@ (:issue:`20204`) * As part of PEP 492 implementation, ``tp_reserved`` slot of - ``PyTypeObject`` was replaced with ``tp_as_async`` slot. + :c:type:`PyTypeObject` was replaced with a + :c:member:`PyTypeObject.tp_as_async` slot.