New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
asyncio.create_subprocess_exec() only works with main event loop #79802
Comments
This fails if there is no current event loop or if that loop isn't running:
This crashes because the main event loop never was created: Traceback (most recent call last):
File "sample.py", line 8, in <module>
loop.run_until_complete(action(loop))
File "/home/sth/devel/cpython.vanilla/Lib/asyncio/base_events.py", line 589, in run_until_complete
return future.result()
File "sample.py", line 4, in action
proc = await asyncio.create_subprocess_exec('echo', loop=loop)
File "/home/sth/devel/cpython.vanilla/Lib/asyncio/subprocess.py", line 213, in create_subprocess_exec
transport, protocol = await loop.subprocess_exec(
File "/home/sth/devel/cpython.vanilla/Lib/asyncio/base_events.py", line 1542, in subprocess_exec
transport = await self._make_subprocess_transport(
File "/home/sth/devel/cpython.vanilla/Lib/asyncio/unix_events.py", line 193, in _make_subprocess_transport
watcher.add_child_handler(transp.get_pid(),
File "/home/sth/devel/cpython.vanilla/Lib/asyncio/unix_events.py", line 924, in add_child_handler
raise RuntimeError(
RuntimeError: Cannot add child handler, the child watcher does not have a loop attached If we do have a current event loop, for example by calling Expected behavior would be that the loop given to create_subprocess_exec() is used to watch the child process. |
Also impacted. A fix I found is to add `watcher.attach_loop(self)` just after `with events.get_child_watcher() as watcher:` in `_make_subprocess_transport` in `asyncio/unix_events.py`. |
New buildbot failure on koobs-freebsd10 that appears related to (includes) this changeset Full log attached |
Also seems to occur on Ubuntu : https://buildbot.python.org/all/#/builders/141/builds/1912/steps/5/logs/stdio karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ ./python -m unittest test.test_asyncio.test_unix_events
...................Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/karthi/cpython/Lib/threading.py", line 923, in _bootstrap_inner
self.run()
File "/home/karthi/cpython/Lib/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/home/karthi/cpython/Lib/test/test_asyncio/test_unix_events.py", line 1833, in f
self.assertIsInstance(watcher, asyncio.SafeChildWatcher)
File "/home/karthi/cpython/Lib/unittest/case.py", line 1330, in assertIsInstance
self.fail(self._formatMessage(msg, standardMsg))
File "/home/karthi/cpython/Lib/unittest/case.py", line 748, in fail
raise self.failureException(msg)
AssertionError: <asyncio.unix_events.ThreadedChildWatcher object at 0x7f00b357d640> is not an instance of <class 'asyncio.unix_events.SafeChildWatcher'>
../home/karthi/cpython/Lib/asyncio/base_events.py:646: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
_warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
............................................................................................... Ran 116 tests in 1.391s OK |
It seems the assertion has to use ThreadedChildWatcher instead of SafeChildWatcher as the default seems to be changed at [0] . If changing assertion fixes the test I am curious why it didn't fail in the primary CI. |
I cannot reproduce this issue locally :( |
When I ran the tests, the watcher I get is a SafeChildWatcher |
Thanks for the report! The failure depends on tests order execution (maybe you use -jN flag?). The failed test doesn't make sense for default ThreadChildWatcher but still valuable for old default SafeChildWatcher. See #13754 for the fix |
Oh, I understand now!
Thanks for the fix, Andrew! :) |
Is it possible to amend the documentation (https://docs.python.org/3.5/library/asyncio-subprocess.html, etc) to include a note that the loop parameter doesn't work? I'm impacted by this bug and lost a few hours to it. |
is there a workaround for earlier Python versions that does not involve patching the standard library? |
You could potentially try using the new default watcher, See cpython/Lib/asyncio/unix_events.py Line 1326 in 4649202
|
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: