This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: asyncio.sock_recv() blocks normal ioloop actions.
Type: behavior Stage:
Components: asyncio Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, socketpair, vstinner, yselivanov
Priority: normal Keywords:

Created on 2015-06-29 17:25 by socketpair, last changed 2022-04-11 14:58 by admin.

Messages (6)
msg245952 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-06-29 17:25
Suppose that program:

====================================
import asyncio
import socket
def receiver(loop):
    (a, b) = socket.socketpair()
    loop.call_later(1, lambda: print('Should be called inside the loop'))
    end = loop.time() + 3
    print('Starting busy receiver')
    while loop.time() < end:
        a.send(b'test')
        yield from loop.sock_recv(b, 65536)
        # yield from asyncio.sleep(0) # <=====================
    print('Busy receiver complete')
    # just not to stop ioloop immediatelly
    yield from asyncio.sleep(0.5)
def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(receiver(loop))
    loop.close()
if __name__ == '__main__':
    main()
====================================

Without asyncio.sleep(0) it will not fire time-delayed calls! If I add asyncio.sleep(0), everything work right. As I think, It is because recv() syscall is always succeeded, and we never return to ioloop (to epoll() I mean).

In other words, it is classical `reader starvation` as mentioned in "man epoll".

It is not documented, that this function may block event loop, in spite of it returns coroutine! I thought that this function will setup EPOLLIN event for socket's FD + call recv() after that. I spent many time to debug program.
msg245957 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-06-29 19:16
You should develop using asyncio debug mode:

https://docs.python.org/dev/library/asyncio-dev.html#asyncio-dev

haypo@selma$ PYTHONASYNCIODEBUG=1 ./python x.py 
Starting busy receiver
Traceback (most recent call last):
  File "x.py", line 21, in <module>
    main()
  File "x.py", line 18, in main
    loop.run_until_complete(receiver(loop))
  File "/home/haypo/prog/python/default/Lib/asyncio/base_events.py", line 341, in run_until_complete
    return future.result()
  File "/home/haypo/prog/python/default/Lib/asyncio/futures.py", line 276, in result
    raise self._exception
  File "/home/haypo/prog/python/default/Lib/asyncio/tasks.py", line 238, in _step
    result = coro.send(value)
  File "x.py", line 11, in receiver
    yield from loop.sock_recv(b, 65536)
  File "/home/haypo/prog/python/default/Lib/asyncio/selector_events.py", line 316, in sock_recv
    raise ValueError("the socket must be non-blocking")
ValueError: the socket must be non-blocking
/home/haypo/prog/python/default/Lib/asyncio/base_events.py:384: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=True>
sys:1: ResourceWarning: unclosed <socket object at 0x7f0b03d7d688>
sys:1: ResourceWarning: unclosed <socket object at 0x7f0b03d7d5f8>
msg245958 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-06-29 19:23
> You should develop using asyncio debug mode:

Maybe we should promote this check to the production mode?
msg245973 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-06-30 02:42
Adding of b.setblocking(0) after socketpair() does not help.
msg245974 - (view) Author: Марк Коренберг (socketpair) * Date: 2015-06-30 02:43
$ PYTHONASYNCIODEBUG=1 ./bug.py
Starting busy receiver
Busy receiver complete
Executing <Task pending coro=<receiver() running at ./bug.py:16> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.4/asyncio/tasks.py:490> cb=[_run_until_complete_cb() at /usr/lib/python3.4/asyncio/base_events.py:123] created at /usr/lib/python3.4/asyncio/base_events.py:296> took 3.001 seconds
Should be called inside the loop
msg245978 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-06-30 06:17
Le lundi 29 juin 2015, Yury Selivanov <report@bugs.python.org> a écrit :
>
>
> > You should develop using asyncio debug mode:
>
> Maybe we should promote this check to the production mode?
>

asyncio must be slow. The check has a cost, I prefer to keep it only in
debug mode. It's probably more a doc issue. We should maybe repeat the info
at the beginning of the asyncio doc.
History
Date User Action Args
2022-04-11 14:58:18adminsetgithub: 68720
2015-06-30 06:17:49vstinnersetmessages: + msg245978
2015-06-30 02:43:46socketpairsetmessages: + msg245974
2015-06-30 02:42:30socketpairsetmessages: + msg245973
2015-06-29 19:23:15yselivanovsetmessages: + msg245958
2015-06-29 19:16:00vstinnersetmessages: + msg245957
2015-06-29 17:25:51socketpaircreate