classification
Title: async process closing after event loop closed
Type: behavior Stage:
Components: asyncio Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: achimnol, asvetlov, kcwu, yselivanov
Priority: normal Keywords:

Created on 2020-07-17 07:38 by kcwu, last changed 2020-07-19 08:50 by achimnol.

Files
File name Uploaded Description Edit
cancel_proc.py kcwu, 2020-07-17 07:38 code to reproduce this issue
Messages (1)
msg373799 - (view) Author: Kuang-che Wu (kcwu) Date: 2020-07-17 07:38
(following code is attached as well)
import asyncio
import time

workaround = False

async def slow_proc():
  proc = await asyncio.create_subprocess_exec('sleep', '10', stdout=asyncio.subprocess.PIPE)
  try:
    return await proc.stdout.read()
  except asyncio.CancelledError:
    if workaround:
      proc.terminate()
      time.sleep(0.1)  # hope the machine is not too busy

async def func():
  try:
    return await asyncio.wait_for(slow_proc(), timeout=0.1)
  except asyncio.TimeoutError:
    return 'timeout'

def main():
  while True:
    print('test')
    asyncio.run(func())

main()
------------------------------------
Run above code, it may work without error message (expected behavior).

However, depends on timing, it may show warning/error messages
case 1.
Loop <_UnixSelectorEventLoop running=False closed=True debug=False> that handles pid 1257652 is closed

case 2.
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fcbcfc66160>
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 126, in __del__
    self.close()
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 104, in close
    proto.pipe.close()
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 536, in close
    self._close(None)
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 560, in _close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
    self._check_closed()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
------------------------------------
Although running tasks will be cancelled when asyncio.run is finishing, subprocess' exit handler may be invoked after the event loop is closed.

In above code, I provided a workaround. However it just mitigates the problem and not really fix the root cause.

This is related to https://bugs.python.org/issue35539

p.s. My test environment is Debian 5.5.17
History
Date User Action Args
2020-07-19 08:50:01achimnolsetnosy: + achimnol
2020-07-17 07:38:23kcwucreate