This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: What's the by-design behavior when os.fork() is invoked in an asyncio loop?
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, gvanrossum, linsm08, yselivanov
Priority: normal Keywords:

Created on 2020-08-24 09:58 by linsm08, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (2)
msg375839 - (view) Author: simin lin (linsm08) Date: 2020-08-24 09:58
I have the same quesion in stackoverflow. Please refer to
https://stackoverflow.com/questions/63558555/whats-the-by-design-behavior-when-os-fork-is-invoked-in-an-asyncio-loop
to get a better format.


Does asyncio work with os.fork()?

Code Snippet 1:

import asyncio
import os

import aiohttp


async def main():
    url = "https://google.com"
    pid = os.fork()
    if pid == 0:
        # child process
        print("in child process")
        await fetch(url)
        print("in child process done")
    else:
        print("in parent process")
        await asyncio.sleep(20)
        print("in parent process done")


async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

if __name__ == "__main__":
    asyncio.run(main())
Code above works fine.

Code Snippet 2:

import asyncio
import os

import aiohttp


async def main():
    url = "https://google.com"
    pid = os.fork()
    if pid == 0:
        # child process
        print("in child process")
        await asyncio.sleep(10)                  # different with code snippet 1
        # await fetch(url)
        print("in child process done")
    else:
        print("in parent process")
        await asyncio.sleep(20)
        print("in parent process done")


async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

if __name__ == "__main__":
    asyncio.run(main())
Code above will raise following exception:

Traceback (most recent call last):
  File "fork_sleep.py", line 28, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "fork_sleep.py", line 13, in main
    await asyncio.sleep(10)                  # different with code snippet 1
  File "/usr/lib/python3.8/asyncio/tasks.py", line 637, in sleep
    loop = events.get_running_loop()
RuntimeError: no running event loop
The reason for the "no running event loop" exception is that function get_running_loop compare the os.getpid() and the pid saved in loop. When they are different, the exception above is raised.

Please refer to the following code from cpython source code.

def get_running_loop():
    """Return the running event loop.  Raise a RuntimeError if there is none.

    This function is thread-specific.
    """
    # NOTE: this function is implemented in C (see _asynciomodule.c)
    loop = _get_running_loop()
    if loop is None:
        raise RuntimeError('no running event loop')
    return loop


def _get_running_loop():
    """Return the running event loop or None.

    This is a low-level function intended to be used by event loops.
    This function is thread-specific.
    """
    # NOTE: this function is implemented in C (see _asynciomodule.c)
    running_loop, pid = _running_loop.loop_pid
    if running_loop is not None and pid == os.getpid():
        return running_loop
So it seems that asyncio event loop works fine in child process if you don't touch the function get_running_loop. My question is, what is the by-design behavior? Why the author check the pid in function _get_running_loop? And what is the solution if you encounter the "no running event loop" in child process.
msg375845 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-08-24 14:27
As you,can tell from thencode this by design. The child,must create a new event loop explicitly,ifmitmwants to use asyncio, and all,existing Futures are off limits.

Fork is dangerous. Don't use unless you understand it.
History
Date User Action Args
2022-04-11 14:59:35adminsetgithub: 85789
2020-08-24 14:27:18gvanrossumsetstatus: open -> closed

nosy: + gvanrossum
messages: + msg375845

resolution: not a bug
stage: resolved
2020-08-24 09:58:16linsm08create