classification
Title: asyncio and ProcessPoolExecutor: OSError on loop.close()
Type: behavior Stage: resolved
Components: asyncio, Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: remind
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, jonafato, mdk, yselivanov
Priority: normal Keywords:

Created on 2018-07-09 01:30 by jonafato, last changed 2018-07-09 16:17 by yselivanov. This issue is now closed.

Messages (3)
msg321297 - (view) Author: Jon Banafato (jonafato) Date: 2018-07-09 01:30
Calling loop.close() on an eventloop configured to use a ProcessPoolExecutor can result in an OSError.

Expected behavior: no exception raised.
Actual behavior: an OSError is raised, and following this call, quit() fails to terminate the process gracefully. 

Example:

Python 3.8.0a0 (default, Jul  7 2018, 14:35:49)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> from concurrent.futures import ProcessPoolExecutor
>>> import os
>>>
>>> loop = asyncio.get_event_loop_policy().new_event_loop()
>>> loop.set_default_executor(ProcessPoolExecutor())
>>> future = loop.run_in_executor(None, os.getpid)
>>> loop.run_until_complete(future)
24697
>>> loop.close()
Exception ignored in: <function ProcessPoolExecutor._start_queue_management_thread.<locals>.weakref_cb at 0x7f52e7f643f0>
Traceback (most recent call last):
  File "/code/Lib/concurrent/futures/process.py", line 567, in weakref_cb
    thread_wakeup.wakeup()
  File "/code/Lib/concurrent/futures/process.py", line 89, in wakeup
    self._writer.send_bytes(b"")
  File "/code/Lib/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/code/Lib/multiprocessing/connection.py", line 136, in _check_closed
Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/code/Lib/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/code/Lib/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/code/Lib/concurrent/futures/process.py", line 368, in _queue_management_worker
    thread_wakeup.clear()
  File "/code/Lib/concurrent/futures/process.py", line 92, in clear
    while self._reader.poll():
  File "/code/Lib/multiprocessing/connection.py", line 255, in poll
    self._check_closed()
  File "/code/Lib/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

    raise OSError("handle is closed")
OSError: handle is closed
msg321299 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2018-07-09 05:36
Can reproduce with 3.7.0 on Linux.
msg321332 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-07-09 16:17
We plan to deprecate and later prohibit setting ProcessPoolExecutor as the default executor.  See https://bugs.python.org/issue34075 for more details.
History
Date User Action Args
2018-07-09 16:17:25yselivanovsetstatus: open -> closed
resolution: remind
messages: + msg321332

stage: resolved
2018-07-09 05:36:55mdksetnosy: + mdk
messages: + msg321299
2018-07-09 01:30:20jonafatocreate